2024-04-27T04:19:03.797 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-27T04:19:03.797 DEBUG:teuthology.run:Teuthology command: teuthology --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} --archive /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699 --owner scheduled_teuthology@teuthology --name teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/orig.config.yaml 2024-04-27T04:19:03.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T04:19:03.916 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699 branch: reef description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7675699' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: reef ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 wait-for-scrub: false ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 rgw: ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast s3tests: slow_backend: true selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 5403a02127f27ec6c3309ef057522ab70cdeb9d2 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 4225 sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 sleep_before_teardown: 0 subset: 0/1 suite: rgw suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 5403a02127f27ec6c3309ef057522ab70cdeb9d2 targets: smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJUUxDBMHKa3h4hU+1Te0s5UfW3KtoI2wf2oGIlmmCaR+z0Qsf/yQ7S7ElSUVqaJSQGhcM1GrSvfbDENSd4FMc= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8CnoVCmRVutcU2qpT9plruGZg54nSzkONz9hWrprdl9G3Vmwr3TY3xjGxYQThhDKrkdssFmKUcOH4MmyDu2fg= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_22:40:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-27T04:19:03.916 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa; will attempt to use it 2024-04-27T04:19:03.917 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa/tasks 2024-04-27T04:19:03.917 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-27T04:19:03.919 INFO:teuthology.task.internal:Checking packages... 2024-04-27T04:19:03.945 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7d2137f0a666c92a9411c1af5c33e4968be54da6' 2024-04-27T04:19:03.945 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T04:19:03.945 INFO:teuthology.packaging:ref: None 2024-04-27T04:19:03.945 INFO:teuthology.packaging:tag: None 2024-04-27T04:19:03.945 INFO:teuthology.packaging:branch: reef 2024-04-27T04:19:03.945 INFO:teuthology.packaging:sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T04:19:03.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-04-27T04:19:04.253 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1260.g5403a021 2024-04-27T04:19:04.255 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-27T04:19:04.261 INFO:teuthology.task.internal:no buildpackages task found 2024-04-27T04:19:04.261 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-27T04:19:04.276 INFO:teuthology.task.internal:Saving configuration 2024-04-27T04:19:04.289 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-27T04:19:04.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T04:19:04.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 04:13:51.975271', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJUUxDBMHKa3h4hU+1Te0s5UfW3KtoI2wf2oGIlmmCaR+z0Qsf/yQ7S7ElSUVqaJSQGhcM1GrSvfbDENSd4FMc='} 2024-04-27T04:19:04.343 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 04:13:51.976935', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8CnoVCmRVutcU2qpT9plruGZg54nSzkONz9hWrprdl9G3Vmwr3TY3xjGxYQThhDKrkdssFmKUcOH4MmyDu2fg='} 2024-04-27T04:19:04.343 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-27T04:19:04.349 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-04-27T04:19:04.349 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-04-27T04:19:04.349 INFO:teuthology.run_tasks:Running task console_log... 2024-04-27T04:19:04.420 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f954fa373a0>, signals=[15]) 2024-04-27T04:19:04.421 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-27T04:19:04.426 INFO:teuthology.task.internal:Opening connections... 2024-04-27T04:19:04.426 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T04:19:04.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:19:04.507 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:19:04.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:19:04.582 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-27T04:19:04.588 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-27T04:19:04.610 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-27T04:19:04.610 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T04:19:04.669 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-27T04:19:04.670 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T04:19:04.670 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-27T04:19:04.694 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-04-27T04:19:04.716 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-04-27T04:19:04.716 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-04-27T04:19:04.777 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-04-27T04:19:04.777 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8" 2024-04-27T04:19:04.777 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-04-27T04:19:04.777 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-27T04:19:04.778 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T04:19:04.778 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-04-27T04:19:04.809 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-27T04:19:04.818 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-27T04:19:04.823 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-27T04:19:04.823 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-27T04:19:04.825 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-04-27T04:19:04.846 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-27T04:19:04.855 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-27T04:19:04.856 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-27T04:19:04.886 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-04-27T04:19:04.918 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-27T04:19:04.975 INFO:teuthology.run_tasks:Running task kernel... 2024-04-27T04:19:04.989 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-27T04:19:04.989 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-27T04:19:04.990 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T04:19:04.990 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T04:19:05.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:19:05.010 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-27T04:19:05.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:19:05.012 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-04-27T04:19:05.072 INFO:teuthology.orchestra.run.smithi138.stdout:4.18.0-489.el8.x86_64 2024-04-27T04:19:05.073 INFO:teuthology.task.kernel:Running kernel on smithi138: 4.18.0-489.el8.x86_64 2024-04-27T04:19:05.073 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-04-27T04:19:05.074 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-489.el8.x86_64 2024-04-27T04:19:05.074 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-489.el8.x86_64 2024-04-27T04:19:05.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2024-04-27T04:19:23.138 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - AppStream 1.8 MB/s | 28 MB 00:15 2024-04-27T04:19:28.974 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - AppStream 1.3 MB/s | 28 MB 00:21 2024-04-27T04:19:31.738 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - BaseOS 2.9 MB/s | 10 MB 00:03 2024-04-27T04:19:33.764 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras 45 kB/s | 18 kB 00:00 2024-04-27T04:19:34.072 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras common packages 38 kB/s | 7.7 kB 00:00 2024-04-27T04:19:34.349 INFO:teuthology.orchestra.run.smithi138.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-04-27T04:19:35.560 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-04-27T04:19:36.132 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - BaseOS 4.7 MB/s | 10 MB 00:02 2024-04-27T04:19:38.266 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras 35 kB/s | 18 kB 00:00 2024-04-27T04:19:38.567 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras common packages 40 kB/s | 7.7 kB 00:00 2024-04-27T04:19:38.844 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-04-27T04:19:39.350 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 542 kB/s | 24 kB 00:00 2024-04-27T04:19:40.245 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-04-27T04:19:42.406 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-27T04:19:42.406 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-27T04:19:42.522 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:19:42.523 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:19:42.523 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T04:19:42.523 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:19:42.523 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout:Install 3 Packages 2024-04-27T04:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:19:42.525 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 90 M 2024-04-27T04:19:42.525 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 96 M 2024-04-27T04:19:42.525 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:19:43.210 INFO:teuthology.orchestra.run.smithi138.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 16 MB/s | 10 MB 00:00 2024-04-27T04:19:44.028 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 560 kB/s | 24 kB 00:00 2024-04-27T04:19:44.185 INFO:teuthology.orchestra.run.smithi138.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 23 MB/s | 36 MB 00:01 2024-04-27T04:19:44.619 INFO:teuthology.orchestra.run.smithi138.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 21 MB/s | 44 MB 00:02 2024-04-27T04:19:44.619 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:19:44.619 INFO:teuthology.orchestra.run.smithi138.stdout:Total 43 MB/s | 90 MB 00:02 2024-04-27T04:19:45.304 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:19:45.729 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:19:45.729 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:19:46.974 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:19:46.992 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:19:47.134 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-27T04:19:47.138 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-27T04:19:47.244 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:19:47.245 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:19:47.245 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-04-27T04:19:47.245 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:19:47.245 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout:Install 3 Packages 2024-04-27T04:19:47.246 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:19:47.247 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 90 M 2024-04-27T04:19:47.247 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 96 M 2024-04-27T04:19:47.247 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:19:48.756 INFO:teuthology.orchestra.run.smithi191.stdout:(1/3): kernel-core-4.18.0-552.el8.x86_64.rpm 33 MB/s | 44 MB 00:01 2024-04-27T04:19:48.964 INFO:teuthology.orchestra.run.smithi191.stdout:(2/3): kernel-4.18.0-552.el8.x86_64.rpm 6.8 MB/s | 10 MB 00:01 2024-04-27T04:19:49.767 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:19:50.323 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-27T04:19:50.481 INFO:teuthology.orchestra.run.smithi191.stdout:(3/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 12 MB/s | 36 MB 00:03 2024-04-27T04:19:50.481 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:19:50.482 INFO:teuthology.orchestra.run.smithi191.stdout:Total 28 MB/s | 90 MB 00:03 2024-04-27T04:19:51.165 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:19:51.587 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:19:51.587 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:19:52.827 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:19:52.853 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:19:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-27T04:19:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-27T04:19:55.625 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:19:56.320 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-27T04:19:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-27T04:19:57.987 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:19:59.910 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-27T04:20:00.344 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-27T04:20:03.782 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-27T04:20:04.075 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:19.904 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:19.930 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:20.276 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:20.276 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-27T04:20:20.277 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-27T04:20:20.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:20.797 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:20:20.797 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:20:20.797 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-27T04:20:20.797 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-27T04:20:20.797 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:20:20.798 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:20:20.951 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T04:20:21.188 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-27T04:20:21.188 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-27T04:20:21.189 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-27T04:20:21.189 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-27T04:20:21.189 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-27T04:20:21.189 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-27T04:20:21.189 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-04-27T04:20:21.189 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T04:20:21.401 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-27T04:20:21.401 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-27T04:20:21.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.02 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 129.el8 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Group : System Environment/Base 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 9265586 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-27T04:20:21.435 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Relocations : (not relocatable) 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : CentOS BuildSystem 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-27T04:20:21.436 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-04-27T04:20:21.437 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:20:21.437 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-04-27T04:20:21.437 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-27T04:20:21.438 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 4.18.0-552.el8.x86_64 2024-04-27T04:20:21.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-27T04:20:23.215 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-04-27T04:20:25.514 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-04-27T04:20:25.516 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-27T04:20:25.544 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.CdYwvDYCp4 2024-04-27T04:20:25.544 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CdYwvDYCp4 2024-04-27T04:20:25.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.CdYwvDYCp4 2024-04-27T04:20:25.779 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.CdYwvDYCp4 is 6KB 2024-04-27T04:20:25.827 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.CdYwvDYCp4 2024-04-27T04:20:25.844 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-04-27T04:20:25.936 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-27T04:20:25.936 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-27T04:20:25.936 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-27T04:20:25.936 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-27T04:20:25.937 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-27T04:20:26.038 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:26.049 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-04-27T04:20:26.063 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:26.423 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:26.424 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-27T04:20:26.424 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-27T04:20:26.962 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-27T04:20:26.962 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:20:26.962 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:20:26.962 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-27T04:20:26.962 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-27T04:20:26.963 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:20:26.963 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:20:27.105 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T04:20:27.328 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-27T04:20:27.329 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-27T04:20:27.329 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-27T04:20:27.329 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-27T04:20:27.329 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-27T04:20:27.329 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-04-27T04:20:27.329 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-04-27T04:20:27.329 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T04:20:27.541 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-27T04:20:27.541 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-27T04:20:27.541 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.02 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 129.el8 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Group : System Environment/Base 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 9265586 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-27T04:20:27.575 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:Relocations : (not relocatable) 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : CentOS BuildSystem 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : CentOS 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:20:27.576 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2024-04-27T04:20:27.577 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-27T04:20:27.577 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 4.18.0-552.el8.x86_64 2024-04-27T04:20:27.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-27T04:20:29.116 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-04-27T04:20:31.147 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-04-27T04:20:31.148 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-04-27T04:20:31.172 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.QubbspFIFE 2024-04-27T04:20:31.173 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QubbspFIFE 2024-04-27T04:20:31.260 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.QubbspFIFE 2024-04-27T04:20:31.378 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.QubbspFIFE is 6KB 2024-04-27T04:20:31.426 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.QubbspFIFE 2024-04-27T04:20:31.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2024-04-27T04:20:31.510 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-27T04:20:31.510 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-27T04:20:31.510 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-27T04:20:31.510 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-27T04:20:31.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-27T04:20:31.668 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-04-27T04:20:56.072 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-27T04:20:56.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T04:20:56.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:21:01.674 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-04-27T04:21:01.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-27T04:21:01.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:21:36.159 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-27T04:21:45.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-27T04:21:45.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:21:56.086 DEBUG:teuthology.orchestra.remote:timed out 2024-04-27T04:22:03.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-27T04:22:05.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T04:22:05.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:22:05.535 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-27T04:22:05.764 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T04:22:05.764 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-552.el8.x86_64"... 2024-04-27T04:22:05.765 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-27T04:22:05.782 INFO:teuthology.orchestra.run.smithi138.stdout:4.18.0-552.el8.x86_64 2024-04-27T04:22:05.782 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-27T04:22:05.782 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-27T04:22:05.782 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-27T04:22:06.783 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-27T04:22:06.784 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T04:22:06.842 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-27T04:22:06.855 DEBUG:teuthology.parallel:result is None 2024-04-27T04:22:15.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-27T04:22:15.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:22:15.998 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-04-27T04:22:16.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-27T04:22:16.279 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "4.18.0-552.el8.x86_64"... 2024-04-27T04:22:16.279 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-04-27T04:22:16.296 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-552.el8.x86_64 2024-04-27T04:22:16.296 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-27T04:22:16.296 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-27T04:22:16.296 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-27T04:22:17.297 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-27T04:22:17.297 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T04:22:17.548 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2024-04-27T04:22:17.576 DEBUG:teuthology.parallel:result is None 2024-04-27T04:22:17.576 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-27T04:22:17.584 INFO:teuthology.task.internal:Creating test directory... 2024-04-27T04:22:17.584 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T04:22:17.587 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T04:22:17.612 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-27T04:22:17.618 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-27T04:22:17.628 INFO:teuthology.task.internal:Creating archive directory... 2024-04-27T04:22:17.628 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T04:22:17.651 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T04:22:17.927 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-27T04:22:17.934 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-27T04:22:17.934 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T04:22:17.937 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T04:22:17.971 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T04:22:17.982 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T04:22:18.555 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T04:22:18.687 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T04:22:18.690 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-27T04:22:18.698 INFO:teuthology.task.internal:Configuring sudo... 2024-04-27T04:22:18.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T04:22:18.701 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T04:22:19.273 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-27T04:22:19.283 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-27T04:22:19.283 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T04:22:19.286 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T04:22:19.321 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T04:22:19.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T04:22:19.456 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T04:22:19.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T04:22:19.545 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:22:19.545 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T04:22:19.613 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T04:22:19.662 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T04:22:19.819 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T04:22:19.861 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T04:22:19.907 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:22:19.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T04:22:19.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-27T04:22:19.978 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-04-27T04:22:20.040 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T04:22:20.085 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T04:22:20.516 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-27T04:22:20.524 INFO:teuthology.task.internal:Starting timer... 2024-04-27T04:22:20.524 INFO:teuthology.run_tasks:Running task pcp... 2024-04-27T04:22:20.534 INFO:teuthology.run_tasks:Running task selinux... 2024-04-27T04:22:20.542 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-27T04:22:20.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-27T04:22:20.645 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-27T04:22:20.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-04-27T04:22:20.739 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-04-27T04:22:20.741 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-27T04:22:20.741 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-27T04:22:20.797 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-27T04:22:20.797 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-04-27T04:22:20.834 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-04-27T04:22:20.834 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-04-27T04:22:20.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T04:22:20.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:22:20.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T04:22:20.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:22:20.901 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-27T04:22:20.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-27T04:22:20.931 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-04-27T04:22:20.978 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-27T04:22:20.992 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-27T04:22:20.999 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-27T04:22:21.125 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-27T04:22:21.141 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-27T04:22:21.143 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi138.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-27T04:29:16.109 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-04-27T04:29:16.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T04:29:16.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:29:16.186 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-27T04:29:16.212 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T04:29:16.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-27T04:29:16.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T04:29:16.288 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-04-27T04:29:16.359 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-27T04:29:16.360 INFO:teuthology.run_tasks:Running task clock... 2024-04-27T04:29:16.370 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-27T04:29:16.370 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T04:29:16.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T04:29:16.373 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T04:29:16.373 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T04:29:16.402 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T04:29:16.416 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T04:29:16.434 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T04:29:16.449 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T04:29:16.484 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-27T04:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-27T04:29:16.508 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T04:29:16.518 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-04-27T04:29:16.521 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T04:29:16.529 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-04-27T04:29:16.543 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T04:29:16.557 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T04:29:16.597 INFO:teuthology.orchestra.run.smithi138.stderr:bash: ntpq: command not found 2024-04-27T04:29:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T04:29:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-27T04:29:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.602 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.616 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2024-04-27T04:29:16.620 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T04:29:16.621 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-04-27T04:29:16.621 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.621 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.621 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.621 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:29:16.621 INFO:teuthology.run_tasks:Running task install... 2024-04-27T04:29:16.629 DEBUG:teuthology.task.install:project ceph 2024-04-27T04:29:16.629 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6'}} 2024-04-27T04:29:16.629 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6'} 2024-04-27T04:29:16.629 INFO:teuthology.task.install:Using flavor: default 2024-04-27T04:29:16.635 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-27T04:29:16.635 INFO:teuthology.task.install:extra packages: [] 2024-04-27T04:29:16.636 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'tag': None, 'wait_for_package': False} 2024-04-27T04:29:16.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T04:29:16.638 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'tag': None, 'wait_for_package': False} 2024-04-27T04:29:16.639 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T04:29:16.865 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/ 2024-04-27T04:29:16.866 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1258.g7d2137f0 2024-04-27T04:29:16.880 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/ 2024-04-27T04:29:16.881 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1258.g7d2137f0 2024-04-27T04:29:16.992 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T04:29:16.992 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:29:16.992 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T04:29:17.019 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T04:29:17.020 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d2137f0a666c92a9411c1af5c33e4968be54da6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T04:29:17.056 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T04:29:17.056 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:29:17.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T04:29:17.084 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T04:29:17.084 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d2137f0a666c92a9411c1af5c33e4968be54da6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T04:29:17.088 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T04:29:17.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T04:29:17.166 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T04:29:17.230 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T04:29:17.232 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-04-27T04:29:17.234 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-04-27T04:29:17.297 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-27T04:29:17.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-27T04:29:17.505 INFO:teuthology.orchestra.run.smithi191.stdout:78 files removed 2024-04-27T04:29:17.533 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw 2024-04-27T04:29:17.568 INFO:teuthology.orchestra.run.smithi138.stdout:78 files removed 2024-04-27T04:29:17.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-04-27T04:29:18.200 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 307 kB/s | 80 kB 00:00 2024-04-27T04:29:18.366 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 219 kB/s | 80 kB 00:00 2024-04-27T04:29:18.567 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-27T04:29:18.726 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-27T04:29:18.926 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2024-04-27T04:29:18.976 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2024-04-27T04:29:54.975 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - AppStream 796 kB/s | 28 MB 00:35 2024-04-27T04:29:59.280 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - AppStream 710 kB/s | 28 MB 00:40 2024-04-27T04:30:02.600 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - BaseOS 3.9 MB/s | 10 MB 00:02 2024-04-27T04:30:08.584 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras 4.1 kB/s | 18 kB 00:04 2024-04-27T04:30:10.451 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras common packages 4.4 kB/s | 7.7 kB 00:01 2024-04-27T04:30:12.577 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - PowerTools 2.4 MB/s | 5.0 MB 00:02 2024-04-27T04:30:13.787 INFO:teuthology.orchestra.run.smithi138.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2024-04-27T04:30:16.491 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 16 MB 00:02 2024-04-27T04:30:20.289 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 366 kB/s | 24 kB 00:00 2024-04-27T04:30:23.029 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - BaseOS 567 kB/s | 10 MB 00:18 2024-04-27T04:30:23.993 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:30:23.995 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:30:23.995 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:30:23.995 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 13 M 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.7 M 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.6 M 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 5.5 M 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 22 M 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 24 k 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 780 k 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-27T04:30:23.996 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 473 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 6.5 M 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 45 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 129 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 197 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 370 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 364 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 107 k 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:30:23.997 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:30:23.998 INFO:teuthology.orchestra.run.smithi138.stdout:Install 21 Packages 2024-04-27T04:30:23.998 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-04-27T04:30:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:30:23.998 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 60 M 2024-04-27T04:30:23.998 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:30:24.898 INFO:teuthology.orchestra.run.smithi138.stdout:(1/23): ceph-base-18.2.2-1258.g7d2137f0.el8.x86 12 MB/s | 5.5 MB 00:00 2024-04-27T04:30:24.940 INFO:teuthology.orchestra.run.smithi138.stdout:(2/23): ceph-selinux-18.2.2-1258.g7d2137f0.el8. 584 kB/s | 24 kB 00:00 2024-04-27T04:30:25.024 INFO:teuthology.orchestra.run.smithi138.stdout:(3/23): libcephfs2-18.2.2-1258.g7d2137f0.el8.x8 9.1 MB/s | 780 kB 00:00 2024-04-27T04:30:25.074 INFO:teuthology.orchestra.run.smithi138.stdout:(4/23): libradosstriper1-18.2.2-1258.g7d2137f0. 9.3 MB/s | 473 kB 00:00 2024-04-27T04:30:25.145 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras 35 kB/s | 18 kB 00:00 2024-04-27T04:30:25.467 INFO:teuthology.orchestra.run.smithi138.stdout:(5/23): librgw2-18.2.2-1258.g7d2137f0.el8.x86_6 17 MB/s | 6.5 MB 00:00 2024-04-27T04:30:25.509 INFO:teuthology.orchestra.run.smithi138.stdout:(6/23): python3-ceph-argparse-18.2.2-1258.g7d21 1.1 MB/s | 45 kB 00:00 2024-04-27T04:30:25.567 INFO:teuthology.orchestra.run.smithi138.stdout:(7/23): python3-ceph-common-18.2.2-1258.g7d2137 2.2 MB/s | 129 kB 00:00 2024-04-27T04:30:25.595 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras common packages 23 kB/s | 7.7 kB 00:00 2024-04-27T04:30:25.634 INFO:teuthology.orchestra.run.smithi138.stdout:(8/23): python3-cephfs-18.2.2-1258.g7d2137f0.el 2.9 MB/s | 197 kB 00:00 2024-04-27T04:30:25.701 INFO:teuthology.orchestra.run.smithi138.stdout:(9/23): python3-rados-18.2.2-1258.g7d2137f0.el8 5.4 MB/s | 370 kB 00:00 2024-04-27T04:30:25.768 INFO:teuthology.orchestra.run.smithi138.stdout:(10/23): python3-rbd-18.2.2-1258.g7d2137f0.el8. 5.3 MB/s | 364 kB 00:00 2024-04-27T04:30:25.835 INFO:teuthology.orchestra.run.smithi138.stdout:(11/23): python3-rgw-18.2.2-1258.g7d2137f0.el8. 1.6 MB/s | 107 kB 00:00 2024-04-27T04:30:26.311 INFO:teuthology.orchestra.run.smithi138.stdout:(12/23): ceph-radosgw-18.2.2-1258.g7d2137f0.el8 7.0 MB/s | 13 MB 00:01 2024-04-27T04:30:26.463 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - PowerTools 6.4 MB/s | 5.0 MB 00:00 2024-04-27T04:30:26.795 INFO:teuthology.orchestra.run.smithi138.stdout:(13/23): librdkafka-1.6.1-1.el8.x86_64.rpm 706 kB/s | 677 kB 00:00 2024-04-27T04:30:26.845 INFO:teuthology.orchestra.run.smithi138.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 486 kB/s | 259 kB 00:00 2024-04-27T04:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 438 kB/s | 44 kB 00:00 2024-04-27T04:30:27.096 INFO:teuthology.orchestra.run.smithi138.stdout:(16/23): ceph-common-18.2.2-1258.g7d2137f0.el8. 8.1 MB/s | 22 MB 00:02 2024-04-27T04:30:27.121 INFO:teuthology.orchestra.run.smithi138.stdout:(17/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 171 kB/s | 47 kB 00:00 2024-04-27T04:30:27.205 INFO:teuthology.orchestra.run.smithi138.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 542 kB/s | 59 kB 00:00 2024-04-27T04:30:27.255 INFO:teuthology.orchestra.run.smithi138.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 852 kB/s | 306 kB 00:00 2024-04-27T04:30:27.297 INFO:teuthology.orchestra.run.smithi138.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 427 kB/s | 75 kB 00:00 2024-04-27T04:30:27.590 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-04-27T04:30:27.622 INFO:teuthology.orchestra.run.smithi138.stdout:(21/23): librbd1-18.2.2-1258.g7d2137f0.el8.x86_ 11 MB/s | 3.6 MB 00:00 2024-04-27T04:30:27.696 INFO:teuthology.orchestra.run.smithi138.stdout:(22/23): librados2-18.2.2-1258.g7d2137f0.el8.x8 8.4 MB/s | 3.7 MB 00:00 2024-04-27T04:30:28.549 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2024-04-27T04:30:28.606 INFO:teuthology.orchestra.run.smithi138.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.2 MB/s | 1.7 MB 00:01 2024-04-27T04:30:28.608 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:30:28.609 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 60 MB 00:04 2024-04-27T04:30:28.652 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:30:28.698 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:30:28.699 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:30:29.055 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:30:29.058 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:30:30.197 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:30:30.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-27T04:30:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2024-04-27T04:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-27T04:30:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-27T04:30:31.991 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/25 2024-04-27T04:30:32.083 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/25 2024-04-27T04:30:32.364 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/25 2024-04-27T04:30:32.374 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 385 kB/s | 24 kB 00:00 2024-04-27T04:30:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/25 2024-04-27T04:30:32.978 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/25 2024-04-27T04:30:33.174 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/25 2024-04-27T04:30:33.229 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/25 2024-04-27T04:30:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-27T04:30:33.639 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-27T04:30:33.811 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 10/25 2024-04-27T04:30:34.043 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 11/25 2024-04-27T04:30:34.284 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 12/25 2024-04-27T04:30:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 13/25 2024-04-27T04:30:34.540 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 13/25 2024-04-27T04:30:34.758 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2024-04-27T04:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2024-04-27T04:30:35.723 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 16/25 2024-04-27T04:30:35.803 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 16/25 2024-04-27T04:30:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 17/25 2024-04-27T04:30:36.087 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:30:36.089 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 13 M 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.7 M 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.6 M 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 5.5 M 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 22 M 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 24 k 2024-04-27T04:30:36.090 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 780 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 473 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 6.5 M 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 45 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 129 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 197 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 370 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 364 k 2024-04-27T04:30:36.091 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 107 k 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout:Install 21 Packages 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 2 Packages 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:30:36.092 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 60 M 2024-04-27T04:30:36.093 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:30:36.217 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-27T04:30:36.407 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 19/25 2024-04-27T04:30:37.050 INFO:teuthology.orchestra.run.smithi191.stdout:(1/23): ceph-base-18.2.2-1258.g7d2137f0.el8.x86 8.4 MB/s | 5.5 MB 00:00 2024-04-27T04:30:37.109 INFO:teuthology.orchestra.run.smithi191.stdout:(2/23): ceph-selinux-18.2.2-1258.g7d2137f0.el8. 418 kB/s | 24 kB 00:00 2024-04-27T04:30:37.226 INFO:teuthology.orchestra.run.smithi191.stdout:(3/23): libcephfs2-18.2.2-1258.g7d2137f0.el8.x8 6.5 MB/s | 780 kB 00:00 2024-04-27T04:30:37.401 INFO:teuthology.orchestra.run.smithi191.stdout:(4/23): ceph-radosgw-18.2.2-1258.g7d2137f0.el8. 13 MB/s | 13 MB 00:01 2024-04-27T04:30:37.426 INFO:teuthology.orchestra.run.smithi191.stdout:(5/23): libradosstriper1-18.2.2-1258.g7d2137f0. 2.3 MB/s | 473 kB 00:00 2024-04-27T04:30:37.518 INFO:teuthology.orchestra.run.smithi191.stdout:(6/23): python3-ceph-argparse-18.2.2-1258.g7d21 487 kB/s | 45 kB 00:00 2024-04-27T04:30:37.610 INFO:teuthology.orchestra.run.smithi191.stdout:(7/23): python3-ceph-common-18.2.2-1258.g7d2137 1.4 MB/s | 129 kB 00:00 2024-04-27T04:30:37.711 INFO:teuthology.orchestra.run.smithi191.stdout:(8/23): python3-cephfs-18.2.2-1258.g7d2137f0.el 1.9 MB/s | 197 kB 00:00 2024-04-27T04:30:37.786 INFO:teuthology.orchestra.run.smithi191.stdout:(9/23): librgw2-18.2.2-1258.g7d2137f0.el8.x86_6 17 MB/s | 6.5 MB 00:00 2024-04-27T04:30:37.811 INFO:teuthology.orchestra.run.smithi191.stdout:(10/23): python3-rados-18.2.2-1258.g7d2137f0.el 3.6 MB/s | 370 kB 00:00 2024-04-27T04:30:37.869 INFO:teuthology.orchestra.run.smithi191.stdout:(11/23): python3-rbd-18.2.2-1258.g7d2137f0.el8. 4.3 MB/s | 364 kB 00:00 2024-04-27T04:30:37.895 INFO:teuthology.orchestra.run.smithi191.stdout:(12/23): python3-rgw-18.2.2-1258.g7d2137f0.el8. 1.2 MB/s | 107 kB 00:00 2024-04-27T04:30:38.503 INFO:teuthology.orchestra.run.smithi191.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 427 kB/s | 259 kB 00:00 2024-04-27T04:30:38.604 INFO:teuthology.orchestra.run.smithi191.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 440 kB/s | 44 kB 00:00 2024-04-27T04:30:38.687 INFO:teuthology.orchestra.run.smithi191.stdout:(15/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-04-27T04:30:38.788 INFO:teuthology.orchestra.run.smithi191.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 3.0 MB/s | 306 kB 00:00 2024-04-27T04:30:38.813 INFO:teuthology.orchestra.run.smithi191.stdout:(17/23): librdkafka-1.6.1-1.el8.x86_64.rpm 717 kB/s | 677 kB 00:00 2024-04-27T04:30:38.838 INFO:teuthology.orchestra.run.smithi191.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2024-04-27T04:30:38.897 INFO:teuthology.orchestra.run.smithi191.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2024-04-27T04:30:39.022 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/25 2024-04-27T04:30:39.147 INFO:teuthology.orchestra.run.smithi191.stdout:(20/23): ceph-common-18.2.2-1258.g7d2137f0.el8. 7.9 MB/s | 22 MB 00:02 2024-04-27T04:30:39.189 INFO:teuthology.orchestra.run.smithi191.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 4.9 MB/s | 1.7 MB 00:00 2024-04-27T04:30:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/25 2024-04-27T04:30:39.273 INFO:teuthology.orchestra.run.smithi191.stdout:(22/23): librados2-18.2.2-1258.g7d2137f0.el8.x8 9.8 MB/s | 3.7 MB 00:00 2024-04-27T04:30:39.557 INFO:teuthology.orchestra.run.smithi191.stdout:(23/23): librbd1-18.2.2-1258.g7d2137f0.el8.x86_ 8.7 MB/s | 3.6 MB 00:00 2024-04-27T04:30:39.559 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:30:39.559 INFO:teuthology.orchestra.run.smithi191.stdout:Total 17 MB/s | 60 MB 00:03 2024-04-27T04:30:39.602 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:30:39.649 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:30:39.649 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:30:39.849 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/25 2024-04-27T04:30:40.001 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:30:40.005 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:30:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 21/25 2024-04-27T04:30:40.282 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 21/25 2024-04-27T04:30:40.448 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 22/25 2024-04-27T04:30:41.220 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:30:41.477 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-27T04:30:41.669 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2024-04-27T04:30:41.854 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-27T04:30:42.252 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-27T04:30:42.454 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/25 2024-04-27T04:30:42.570 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/25 2024-04-27T04:30:42.860 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/25 2024-04-27T04:30:43.054 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/25 2024-04-27T04:30:43.496 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/25 2024-04-27T04:30:43.720 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/25 2024-04-27T04:30:43.787 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/25 2024-04-27T04:30:43.996 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-27T04:30:44.202 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-27T04:30:44.390 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 10/25 2024-04-27T04:30:44.589 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 11/25 2024-04-27T04:30:44.813 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 12/25 2024-04-27T04:30:44.987 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 13/25 2024-04-27T04:30:45.053 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 13/25 2024-04-27T04:30:45.280 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2024-04-27T04:30:46.076 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2024-04-27T04:30:46.237 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 16/25 2024-04-27T04:30:46.311 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 16/25 2024-04-27T04:30:46.536 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 17/25 2024-04-27T04:30:46.771 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-27T04:30:47.012 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 19/25 2024-04-27T04:30:49.553 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/25 2024-04-27T04:30:49.804 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/25 2024-04-27T04:30:50.439 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/25 2024-04-27T04:30:50.713 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 21/25 2024-04-27T04:30:50.816 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 21/25 2024-04-27T04:30:51.011 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 22/25 2024-04-27T04:31:00.107 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 22/25 2024-04-27T04:31:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-27T04:31:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-27T04:31:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2024-04-27T04:31:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /run 2024-04-27T04:31:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-27T04:31:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-27T04:31:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-27T04:31:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-27T04:31:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2024-04-27T04:31:00.108 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:01.450 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 23/25 2024-04-27T04:31:01.471 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 23/25 2024-04-27T04:31:01.471 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-27T04:31:01.722 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-27T04:31:01.722 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-27T04:31:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-27T04:31:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/25 2024-04-27T04:31:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 6/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 8/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 9/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 10/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 11/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 12/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 13/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-27T04:31:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 22/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-27T04:31:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 24/25 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:03.405 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-27T04:31:03.405 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:03.405 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:03.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-04-27T04:31:04.465 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:44 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:06.116 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 47 M 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-27T04:31:06.117 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 142 k 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-04-27T04:31:06.118 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:06.119 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 48 M 2024-04-27T04:31:06.119 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 213 M 2024-04-27T04:31:06.119 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:31:06.490 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): libcephsqlite-18.2.2-1258.g7d2137f0.el8. 762 kB/s | 142 kB 00:00 2024-04-27T04:31:06.832 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 384 kB/s | 203 kB 00:00 2024-04-27T04:31:06.941 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 418 kB/s | 188 kB 00:00 2024-04-27T04:31:07.116 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 404 kB/s | 71 kB 00:00 2024-04-27T04:31:07.283 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 717 kB/s | 323 kB 00:00 2024-04-27T04:31:09.891 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): ceph-test-18.2.2-1258.g7d2137f0.el8.x86_ 13 MB/s | 47 MB 00:03 2024-04-27T04:31:09.893 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:09.893 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 48 MB 00:03 2024-04-27T04:31:09.935 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:31:09.956 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:31:09.956 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:31:10.277 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:31:10.280 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:31:10.658 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 22/25 2024-04-27T04:31:10.658 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2024-04-27T04:31:10.659 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:10.746 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:31:10.990 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-27T04:31:11.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-27T04:31:11.335 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-27T04:31:11.413 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-27T04:31:11.598 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-27T04:31:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/6 2024-04-27T04:31:12.042 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 23/25 2024-04-27T04:31:12.063 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 23/25 2024-04-27T04:31:12.063 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-27T04:31:12.301 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-27T04:31:12.301 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-27T04:31:12.997 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-27T04:31:12.997 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 6/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 8/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 9/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 10/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 11/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 12/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 13/25 2024-04-27T04:31:12.998 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 22/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-27T04:31:12.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 24/25 2024-04-27T04:31:13.944 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-27T04:31:13.944 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:13.944 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.945 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-27T04:31:13.946 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.947 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.947 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:13.947 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-27T04:31:13.947 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:13.947 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:31:14.322 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-test 2024-04-27T04:31:15.050 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:43 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:31:16.328 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/6 2024-04-27T04:31:16.549 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/6 2024-04-27T04:31:16.693 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 47 M 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 142 k 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-27T04:31:16.694 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout:Install 6 Packages 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 48 M 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 213 M 2024-04-27T04:31:16.695 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:31:16.833 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/6 2024-04-27T04:31:16.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/6 2024-04-27T04:31:16.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/6 2024-04-27T04:31:16.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-27T04:31:16.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-27T04:31:16.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-27T04:31:17.069 INFO:teuthology.orchestra.run.smithi191.stdout:(1/6): libcephsqlite-18.2.2-1258.g7d2137f0.el8. 756 kB/s | 142 kB 00:00 2024-04-27T04:31:17.302 INFO:teuthology.orchestra.run.smithi191.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 482 kB/s | 203 kB 00:00 2024-04-27T04:31:17.314 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: jq-1.6-8.el8.x86_64 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:17.315 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:17.519 INFO:teuthology.orchestra.run.smithi191.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 418 kB/s | 188 kB 00:00 2024-04-27T04:31:17.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-04-27T04:31:17.586 INFO:teuthology.orchestra.run.smithi191.stdout:(4/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.1 MB/s | 323 kB 00:00 2024-04-27T04:31:17.619 INFO:teuthology.orchestra.run.smithi191.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 704 kB/s | 71 kB 00:00 2024-04-27T04:31:18.276 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:58 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:19.950 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 6.4 k 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 2.3 M 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 1.6 M 2024-04-27T04:31:19.953 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 245 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 4.8 M 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 18 M 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-27T04:31:19.954 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-27T04:31:19.955 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout:Install 32 Packages 2024-04-27T04:31:19.956 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:19.957 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 31 M 2024-04-27T04:31:19.957 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 124 M 2024-04-27T04:31:19.957 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:31:20.061 INFO:teuthology.orchestra.run.smithi191.stdout:(6/6): ceph-test-18.2.2-1258.g7d2137f0.el8.x86_ 15 MB/s | 47 MB 00:03 2024-04-27T04:31:20.063 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:20.063 INFO:teuthology.orchestra.run.smithi191.stdout:Total 14 MB/s | 48 MB 00:03 2024-04-27T04:31:20.105 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:31:20.126 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:31:20.126 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:31:20.465 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:31:20.468 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:31:20.531 INFO:teuthology.orchestra.run.smithi138.stdout:(1/32): ceph-18.2.2-1258.g7d2137f0.el8.x86_64.r 51 kB/s | 6.4 kB 00:00 2024-04-27T04:31:20.764 INFO:teuthology.orchestra.run.smithi138.stdout:(2/32): ceph-mds-18.2.2-1258.g7d2137f0.el8.x86_ 6.3 MB/s | 2.3 MB 00:00 2024-04-27T04:31:20.798 INFO:teuthology.orchestra.run.smithi138.stdout:(3/32): ceph-mgr-18.2.2-1258.g7d2137f0.el8.x86_ 4.0 MB/s | 1.6 MB 00:00 2024-04-27T04:31:20.898 INFO:teuthology.orchestra.run.smithi138.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1258.g7d21 2.4 MB/s | 245 kB 00:00 2024-04-27T04:31:20.947 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:31:21.178 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-27T04:31:21.307 INFO:teuthology.orchestra.run.smithi138.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 384 kB/s | 157 kB 00:00 2024-04-27T04:31:21.410 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-27T04:31:21.410 INFO:teuthology.orchestra.run.smithi138.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 387 kB/s | 39 kB 00:00 2024-04-27T04:31:21.541 INFO:teuthology.orchestra.run.smithi138.stdout:(7/32): ceph-mon-18.2.2-1258.g7d2137f0.el8.x86_ 4.8 MB/s | 4.8 MB 00:01 2024-04-27T04:31:21.566 INFO:teuthology.orchestra.run.smithi138.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 652 kB/s | 103 kB 00:00 2024-04-27T04:31:21.589 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-27T04:31:21.665 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-27T04:31:21.767 INFO:teuthology.orchestra.run.smithi138.stdout:(9/32): ceph-osd-18.2.2-1258.g7d2137f0.el8.x86_ 19 MB/s | 18 MB 00:00 2024-04-27T04:31:21.876 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-27T04:31:21.884 INFO:teuthology.orchestra.run.smithi138.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.1 MB/s | 249 kB 00:00 2024-04-27T04:31:21.942 INFO:teuthology.orchestra.run.smithi138.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.2 MB/s | 457 kB 00:00 2024-04-27T04:31:21.993 INFO:teuthology.orchestra.run.smithi138.stdout:(12/32): python3-cryptography-3.2.1-7.el8.x86_6 5.8 MB/s | 641 kB 00:00 2024-04-27T04:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout:(13/32): python3-pytz-2017.2-11.el8.noarch.rpm 119 kB/s | 57 kB 00:00 2024-04-27T04:31:22.031 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/6 2024-04-27T04:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 1.1 MB/s | 109 kB 00:00 2024-04-27T04:31:22.419 INFO:teuthology.orchestra.run.smithi138.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 102 kB/s | 44 kB 00:00 2024-04-27T04:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 352 kB/s | 185 kB 00:00 2024-04-27T04:31:22.595 INFO:teuthology.orchestra.run.smithi138.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 313 kB/s | 173 kB 00:00 2024-04-27T04:31:22.645 INFO:teuthology.orchestra.run.smithi138.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2024-04-27T04:31:22.704 INFO:teuthology.orchestra.run.smithi138.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 162 kB/s | 18 kB 00:00 2024-04-27T04:31:22.754 INFO:teuthology.orchestra.run.smithi138.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 449 kB/s | 49 kB 00:00 2024-04-27T04:31:22.830 INFO:teuthology.orchestra.run.smithi138.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 937 kB/s | 384 kB 00:00 2024-04-27T04:31:22.855 INFO:teuthology.orchestra.run.smithi138.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 389 kB/s | 59 kB 00:00 2024-04-27T04:31:22.930 INFO:teuthology.orchestra.run.smithi138.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2024-04-27T04:31:22.964 INFO:teuthology.orchestra.run.smithi138.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 116 kB/s | 16 kB 00:00 2024-04-27T04:31:22.989 INFO:teuthology.orchestra.run.smithi138.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 140 kB/s | 19 kB 00:00 2024-04-27T04:31:23.040 INFO:teuthology.orchestra.run.smithi138.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 219 kB/s | 24 kB 00:00 2024-04-27T04:31:23.073 INFO:teuthology.orchestra.run.smithi138.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-04-27T04:31:23.107 INFO:teuthology.orchestra.run.smithi138.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 234 kB/s | 27 kB 00:00 2024-04-27T04:31:23.174 INFO:teuthology.orchestra.run.smithi138.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.9 MB/s | 255 kB 00:00 2024-04-27T04:31:23.241 INFO:teuthology.orchestra.run.smithi138.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2024-04-27T04:31:23.266 INFO:teuthology.orchestra.run.smithi138.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 537 kB/s | 85 kB 00:00 2024-04-27T04:31:23.291 INFO:teuthology.orchestra.run.smithi138.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 196 kB/s | 23 kB 00:00 2024-04-27T04:31:23.296 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:23.296 INFO:teuthology.orchestra.run.smithi138.stdout:Total 9.3 MB/s | 31 MB 00:03 2024-04-27T04:31:23.376 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:31:23.414 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:31:23.414 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:31:23.738 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:31:23.741 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:31:24.832 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:31:25.071 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-27T04:31:25.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-27T04:31:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-27T04:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-27T04:31:25.912 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-27T04:31:26.101 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-27T04:31:26.327 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-27T04:31:26.522 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-27T04:31:26.619 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/6 2024-04-27T04:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-27T04:31:26.928 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/6 2024-04-27T04:31:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-27T04:31:27.212 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/6 2024-04-27T04:31:27.212 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/6 2024-04-27T04:31:27.212 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/6 2024-04-27T04:31:27.212 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-27T04:31:27.212 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-27T04:31:27.212 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-27T04:31:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-27T04:31:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-8.el8.x86_64 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-27T04:31:27.646 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:27.647 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:31:27.847 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-27T04:31:27.901 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph 2024-04-27T04:31:28.118 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-27T04:31:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-27T04:31:28.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-27T04:31:28.636 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:56 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:31:28.766 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-27T04:31:29.096 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-27T04:31:29.319 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-27T04:31:29.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-27T04:31:29.712 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-27T04:31:29.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-27T04:31:30.187 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-27T04:31:30.294 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:31:30.298 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:30.298 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:31:30.298 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:30.298 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 6.4 k 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 2.3 M 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 1.6 M 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 245 k 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 4.8 M 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 18 M 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-27T04:31:30.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-27T04:31:30.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout:Install 32 Packages 2024-04-27T04:31:30.301 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:30.302 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 31 M 2024-04-27T04:31:30.302 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 124 M 2024-04-27T04:31:30.302 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:31:30.446 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-27T04:31:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-27T04:31:31.070 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-27T04:31:31.154 INFO:teuthology.orchestra.run.smithi191.stdout:(1/32): ceph-18.2.2-1258.g7d2137f0.el8.x86_64.r 36 kB/s | 6.4 kB 00:00 2024-04-27T04:31:31.313 INFO:teuthology.orchestra.run.smithi191.stdout:(2/32): ceph-mgr-18.2.2-1258.g7d2137f0.el8.x86_ 4.6 MB/s | 1.6 MB 00:00 2024-04-27T04:31:31.396 INFO:teuthology.orchestra.run.smithi191.stdout:(3/32): ceph-mds-18.2.2-1258.g7d2137f0.el8.x86_ 5.4 MB/s | 2.3 MB 00:00 2024-04-27T04:31:31.442 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 27/32 2024-04-27T04:31:31.488 INFO:teuthology.orchestra.run.smithi191.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1258.g7d21 2.6 MB/s | 245 kB 00:00 2024-04-27T04:31:31.637 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T04:31:31.872 INFO:teuthology.orchestra.run.smithi191.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 410 kB/s | 157 kB 00:00 2024-04-27T04:31:31.972 INFO:teuthology.orchestra.run.smithi191.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 389 kB/s | 39 kB 00:00 2024-04-27T04:31:32.230 INFO:teuthology.orchestra.run.smithi191.stdout:(7/32): ceph-osd-18.2.2-1258.g7d2137f0.el8.x86_ 20 MB/s | 18 MB 00:00 2024-04-27T04:31:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 365 kB/s | 103 kB 00:00 2024-04-27T04:31:32.531 INFO:teuthology.orchestra.run.smithi191.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 1.6 MB/s | 457 kB 00:00 2024-04-27T04:31:32.565 INFO:teuthology.orchestra.run.smithi191.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 170 kB/s | 57 kB 00:00 2024-04-27T04:31:32.673 INFO:teuthology.orchestra.run.smithi191.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.7 MB/s | 249 kB 00:00 2024-04-27T04:31:32.715 INFO:teuthology.orchestra.run.smithi191.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 2.6 MB/s | 109 kB 00:00 2024-04-27T04:31:32.749 INFO:teuthology.orchestra.run.smithi191.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 3.4 MB/s | 641 kB 00:00 2024-04-27T04:31:32.966 INFO:teuthology.orchestra.run.smithi191.stdout:(14/32): ceph-mon-18.2.2-1258.g7d2137f0.el8.x86 2.7 MB/s | 4.8 MB 00:01 2024-04-27T04:31:33.133 INFO:teuthology.orchestra.run.smithi191.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 104 kB/s | 44 kB 00:00 2024-04-27T04:31:33.267 INFO:teuthology.orchestra.run.smithi191.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 358 kB/s | 185 kB 00:00 2024-04-27T04:31:33.286 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T04:31:33.367 INFO:teuthology.orchestra.run.smithi191.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-04-27T04:31:33.415 INFO:teuthology.orchestra.run.smithi191.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 384 kB/s | 173 kB 00:00 2024-04-27T04:31:33.467 INFO:teuthology.orchestra.run.smithi191.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.1 MB/s | 384 kB 00:00 2024-04-27T04:31:33.486 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T04:31:33.492 INFO:teuthology.orchestra.run.smithi191.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 140 kB/s | 18 kB 00:00 2024-04-27T04:31:33.568 INFO:teuthology.orchestra.run.smithi191.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 319 kB/s | 49 kB 00:00 2024-04-27T04:31:33.593 INFO:teuthology.orchestra.run.smithi191.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 466 kB/s | 59 kB 00:00 2024-04-27T04:31:33.710 INFO:teuthology.orchestra.run.smithi191.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2024-04-27T04:31:33.744 INFO:teuthology.orchestra.run.smithi191.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 125 kB/s | 19 kB 00:00 2024-04-27T04:31:33.778 INFO:teuthology.orchestra.run.smithi191.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 993 kB/s | 283 kB 00:00 2024-04-27T04:31:33.811 INFO:teuthology.orchestra.run.smithi191.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 237 kB/s | 24 kB 00:00 2024-04-27T04:31:33.853 INFO:teuthology.orchestra.run.smithi191.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-04-27T04:31:33.895 INFO:teuthology.orchestra.run.smithi191.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 235 kB/s | 27 kB 00:00 2024-04-27T04:31:33.963 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T04:31:34.012 INFO:teuthology.orchestra.run.smithi191.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2024-04-27T04:31:34.037 INFO:teuthology.orchestra.run.smithi191.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.1 MB/s | 255 kB 00:00 2024-04-27T04:31:34.063 INFO:teuthology.orchestra.run.smithi191.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 511 kB/s | 85 kB 00:00 2024-04-27T04:31:34.121 INFO:teuthology.orchestra.run.smithi191.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 211 kB/s | 23 kB 00:00 2024-04-27T04:31:34.122 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:34.123 INFO:teuthology.orchestra.run.smithi191.stdout:Total 8.1 MB/s | 31 MB 00:03 2024-04-27T04:31:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T04:31:34.203 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:31:34.241 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:31:34.241 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:31:34.361 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T04:31:34.556 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/32 2024-04-27T04:31:34.560 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:31:34.563 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:31:34.574 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/32 2024-04-27T04:31:34.697 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/32 2024-04-27T04:31:35.095 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/32 2024-04-27T04:31:35.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/32 2024-04-27T04:31:35.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 6/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-27T04:31:35.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-27T04:31:35.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-27T04:31:35.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-27T04:31:35.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-27T04:31:35.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-27T04:31:35.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-27T04:31:35.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-27T04:31:35.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-27T04:31:35.794 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:31:36.017 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-27T04:31:36.261 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-27T04:31:36.495 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-27T04:31:36.726 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-27T04:31:36.760 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-27T04:31:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-2.6-1.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-27T04:31:36.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:36.763 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:36.957 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-27T04:31:37.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-04-27T04:31:37.112 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-27T04:31:37.322 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-27T04:31:37.484 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-27T04:31:37.752 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-27T04:31:37.820 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:17 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:38.021 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-27T04:31:38.293 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-27T04:31:38.567 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-27T04:31:38.965 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-27T04:31:39.272 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-27T04:31:39.439 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:31:39.465 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-27T04:31:39.487 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:39.488 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:31:39.488 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:39.599 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-04-27T04:31:39.676 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-27T04:31:39.878 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-27T04:31:40.244 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-27T04:31:40.351 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:20 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:40.466 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-27T04:31:40.713 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-27T04:31:40.924 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-27T04:31:41.168 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-27T04:31:41.458 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-27T04:31:41.698 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-27T04:31:42.008 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:42.009 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:42.009 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:31:42.009 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:42.009 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:31:42.009 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 170 k 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 170 k 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 692 k 2024-04-27T04:31:42.010 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:31:42.047 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-27T04:31:42.292 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-18.2.2-1258.g7d2137f0.el8.noarch.rpm 604 kB/s | 170 kB 00:00 2024-04-27T04:31:42.292 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:42.292 INFO:teuthology.orchestra.run.smithi138.stdout:Total 602 kB/s | 170 kB 00:00 2024-04-27T04:31:42.293 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:31:42.300 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:31:42.300 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:31:42.303 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:31:42.306 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:31:42.350 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:31:42.383 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-27T04:31:42.791 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 27/32 2024-04-27T04:31:42.885 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:43.020 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T04:31:43.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:43.254 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:43.620 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:43.620 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:43.621 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:31:43.621 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:43.621 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:43.621 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:43.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T04:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:24 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:44.676 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T04:31:44.861 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T04:31:45.338 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T04:31:45.889 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T04:31:46.145 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T04:31:46.281 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:46.281 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 165 k 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:31:46.282 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:46.283 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 165 k 2024-04-27T04:31:46.283 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 503 k 2024-04-27T04:31:46.283 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:31:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/32 2024-04-27T04:31:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/32 2024-04-27T04:31:46.618 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/32 2024-04-27T04:31:46.891 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-18.2.2-1258.g7d2137 272 kB/s | 165 kB 00:00 2024-04-27T04:31:46.892 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:46.892 INFO:teuthology.orchestra.run.smithi138.stdout:Total 271 kB/s | 165 kB 00:00 2024-04-27T04:31:46.892 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:31:46.905 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:31:46.906 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:31:46.956 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:31:46.962 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/32 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/32 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/32 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/32 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/32 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T04:31:47.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 6/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-27T04:31:47.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-27T04:31:47.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-27T04:31:47.851 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-27T04:31:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-27T04:31:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:47.854 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:31:47.884 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:31:48.109 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-base 2024-04-27T04:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T04:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T04:31:48.867 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:16 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:31:49.163 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T04:31:49.163 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:49.163 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:31:49.163 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:49.163 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:49.163 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:49.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-04-27T04:31:50.158 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:30 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:50.469 INFO:teuthology.orchestra.run.smithi191.stdout:Package ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:31:50.516 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:31:50.517 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:31:50.518 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:31:50.624 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install cephadm 2024-04-27T04:31:51.391 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:19 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:31:51.759 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:31:51.807 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:51.808 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:31:51.808 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:51.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T04:31:52.671 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:32 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:53.057 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 170 k 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:53.058 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:31:53.059 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:53.059 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 170 k 2024-04-27T04:31:53.059 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 692 k 2024-04-27T04:31:53.059 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:31:53.291 INFO:teuthology.orchestra.run.smithi191.stdout:cephadm-18.2.2-1258.g7d2137f0.el8.noarch.rpm 735 kB/s | 170 kB 00:00 2024-04-27T04:31:53.291 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:53.291 INFO:teuthology.orchestra.run.smithi191.stdout:Total 732 kB/s | 170 kB 00:00 2024-04-27T04:31:53.292 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:31:53.299 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:31:53.299 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:31:53.301 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:31:53.305 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:31:53.356 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:31:53.907 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:54.039 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:54.269 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:54.330 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 3.5 M 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 24 k 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 15 k 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-27T04:31:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout:Install 5 Packages 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.8 M 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 87 M 2024-04-27T04:31:54.335 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:31:54.665 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T04:31:54.665 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:54.665 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:31:54.665 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:54.665 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:54.665 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:31:54.863 INFO:teuthology.orchestra.run.smithi138.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1258.g7d21 102 kB/s | 15 kB 00:00 2024-04-27T04:31:54.888 INFO:teuthology.orchestra.run.smithi138.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1258.g7d2 139 kB/s | 24 kB 00:00 2024-04-27T04:31:54.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T04:31:55.122 INFO:teuthology.orchestra.run.smithi138.stdout:(3/5): ceph-mgr-dashboard-18.2.2-1258.g7d2137f0 8.6 MB/s | 3.5 MB 00:00 2024-04-27T04:31:55.322 INFO:teuthology.orchestra.run.smithi138.stdout:(4/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 73 kB/s | 34 kB 00:00 2024-04-27T04:31:55.630 INFO:teuthology.orchestra.run.smithi138.stdout:(5/5): python3-routes-2.4.1-12.el8.noarch.rpm 265 kB/s | 196 kB 00:00 2024-04-27T04:31:55.634 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:55.634 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.9 MB/s | 3.8 MB 00:01 2024-04-27T04:31:55.634 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:31:55.642 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:31:55.642 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:31:55.680 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:23 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:31:55.726 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:31:55.729 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:31:55.958 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:31:56.167 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-27T04:31:56.397 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-27T04:31:56.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T04:31:57.322 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repo Size 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 165 k 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:31:57.323 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:31:57.324 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:31:57.324 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:57.324 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 165 k 2024-04-27T04:31:57.324 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 503 k 2024-04-27T04:31:57.324 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:31:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 4/5 2024-04-27T04:31:57.907 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T04:31:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T04:31:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 1/5 2024-04-27T04:31:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 2/5 2024-04-27T04:31:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T04:31:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-27T04:31:58.255 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-immutable-object-cache-18.2.2-1258.g7d2137 178 kB/s | 165 kB 00:00 2024-04-27T04:31:58.257 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:31:58.257 INFO:teuthology.orchestra.run.smithi191.stdout:Total 177 kB/s | 165 kB 00:00 2024-04-27T04:31:58.257 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:31:58.270 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:31:58.270 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:31:58.321 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:31:58.325 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-27T04:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-27T04:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:31:58.835 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:31:58.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T04:31:59.041 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T04:31:59.443 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T04:31:59.662 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:39 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:31:59.811 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T04:31:59.812 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:59.812 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:31:59.812 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:31:59.812 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:31:59.812 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:00.082 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr 2024-04-27T04:32:00.850 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:28 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:01.324 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================================ 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================================ 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 7.4 M 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 8.5.0-22.el8 baseos 645 k 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 8.5.0-22.el8 baseos 172 k 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-27T04:32:01.326 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================================ 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout:Install 8 Packages 2024-04-27T04:32:01.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:01.328 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 36 M 2024-04-27T04:32:01.328 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 210 M 2024-04-27T04:32:01.328 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:02.272 INFO:teuthology.orchestra.run.smithi138.stdout:(1/8): ceph-mgr-diskprediction-local-18.2.2-125 10 MB/s | 7.4 MB 00:00 2024-04-27T04:32:02.448 INFO:teuthology.orchestra.run.smithi191.stdout:Package ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:32:02.495 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:02.497 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:32:02.497 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:02.599 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T04:32:03.351 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:31 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:05.043 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:05.044 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:05.044 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:32:05.044 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:05.044 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:32:05.044 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 3.5 M 2024-04-27T04:32:05.044 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 24 k 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 15 k 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout:Install 5 Packages 2024-04-27T04:32:05.045 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:05.046 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 3.8 M 2024-04-27T04:32:05.046 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 87 M 2024-04-27T04:32:05.046 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:32:05.141 INFO:teuthology.orchestra.run.smithi138.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 1.3 MB/s | 4.7 MB 00:03 2024-04-27T04:32:05.208 INFO:teuthology.orchestra.run.smithi138.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 1.3 MB/s | 4.9 MB 00:03 2024-04-27T04:32:05.308 INFO:teuthology.orchestra.run.smithi138.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.3 MB/s | 225 kB 00:00 2024-04-27T04:32:05.333 INFO:teuthology.orchestra.run.smithi191.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1258.g7d21 105 kB/s | 15 kB 00:00 2024-04-27T04:32:05.350 INFO:teuthology.orchestra.run.smithi191.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1258.g7d2 150 kB/s | 24 kB 00:00 2024-04-27T04:32:05.417 INFO:teuthology.orchestra.run.smithi138.stdout:(5/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 1.2 MB/s | 3.7 MB 00:03 2024-04-27T04:32:05.492 INFO:teuthology.orchestra.run.smithi138.stdout:(6/8): libgfortran-8.5.0-22.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-04-27T04:32:05.576 INFO:teuthology.orchestra.run.smithi191.stdout:(3/5): ceph-mgr-dashboard-18.2.2-1258.g7d2137f0 9.1 MB/s | 3.5 MB 00:00 2024-04-27T04:32:05.592 INFO:teuthology.orchestra.run.smithi138.stdout:(7/8): libquadmath-8.5.0-22.el8.x86_64.rpm 984 kB/s | 172 kB 00:00 2024-04-27T04:32:05.617 INFO:teuthology.orchestra.run.smithi191.stdout:(4/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 118 kB/s | 34 kB 00:00 2024-04-27T04:32:05.759 INFO:teuthology.orchestra.run.smithi191.stdout:(5/5): python3-routes-2.4.1-12.el8.noarch.rpm 480 kB/s | 196 kB 00:00 2024-04-27T04:32:05.761 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:05.761 INFO:teuthology.orchestra.run.smithi191.stdout:Total 5.3 MB/s | 3.8 MB 00:00 2024-04-27T04:32:05.762 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:32:05.770 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:32:05.770 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:32:05.857 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:32:05.860 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:32:06.097 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:32:06.271 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-27T04:32:06.500 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-27T04:32:06.651 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T04:32:07.808 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 4/5 2024-04-27T04:32:08.017 INFO:teuthology.orchestra.run.smithi138.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 5.0 MB/s | 14 MB 00:02 2024-04-27T04:32:08.018 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:08.018 INFO:teuthology.orchestra.run.smithi138.stdout:Total 5.4 MB/s | 36 MB 00:06 2024-04-27T04:32:08.048 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T04:32:08.252 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:08.269 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:08.269 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:08.334 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T04:32:08.334 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 1/5 2024-04-27T04:32:08.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 2/5 2024-04-27T04:32:08.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T04:32:08.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-27T04:32:08.682 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:08.684 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:08.720 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:08.721 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:08.871 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T04:32:09.232 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:09.393 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-8.5.0-22.el8.x86_64 1/8 2024-04-27T04:32:09.524 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libquadmath-8.5.0-22.el8.x86_64 1/8 2024-04-27T04:32:09.621 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:37 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:09.701 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-8.5.0-22.el8.x86_64 2/8 2024-04-27T04:32:10.299 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libgfortran-8.5.0-22.el8.x86_64 2/8 2024-04-27T04:32:10.676 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-27T04:32:11.267 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 7.4 M 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.5.0-22.el8 baseos 645 k 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.5.0-22.el8 baseos 172 k 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-27T04:32:11.269 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout:Install 8 Packages 2024-04-27T04:32:11.270 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:11.271 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 36 M 2024-04-27T04:32:11.271 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 210 M 2024-04-27T04:32:11.271 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:32:11.275 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-27T04:32:11.442 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-27T04:32:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-27T04:32:12.191 INFO:teuthology.orchestra.run.smithi191.stdout:(1/8): ceph-mgr-diskprediction-local-18.2.2-125 14 MB/s | 7.4 MB 00:00 2024-04-27T04:32:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-27T04:32:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-27T04:32:13.116 INFO:teuthology.orchestra.run.smithi191.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 3.3 MB/s | 4.9 MB 00:01 2024-04-27T04:32:13.216 INFO:teuthology.orchestra.run.smithi191.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.2 MB/s | 225 kB 00:00 2024-04-27T04:32:13.333 INFO:teuthology.orchestra.run.smithi191.stdout:(4/8): openblas-0.3.15-4.el8.x86_64.rpm 2.8 MB/s | 4.7 MB 00:01 2024-04-27T04:32:13.500 INFO:teuthology.orchestra.run.smithi191.stdout:(5/8): libgfortran-8.5.0-22.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2024-04-27T04:32:13.584 INFO:teuthology.orchestra.run.smithi191.stdout:(6/8): libquadmath-8.5.0-22.el8.x86_64.rpm 2.0 MB/s | 172 kB 00:00 2024-04-27T04:32:14.463 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-27T04:32:15.019 INFO:teuthology.orchestra.run.smithi191.stdout:(7/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 1.3 MB/s | 3.7 MB 00:02 2024-04-27T04:32:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-27T04:32:15.891 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f 8/8 2024-04-27T04:32:16.210 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f 8/8 2024-04-27T04:32:16.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f 1/8 2024-04-27T04:32:16.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-27T04:32:16.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-27T04:32:16.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-27T04:32:16.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-27T04:32:16.211 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-27T04:32:16.211 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-8.5.0-22.el8.x86_64 7/8 2024-04-27T04:32:16.651 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-8.5.0-22.el8.x86_64 8/8 2024-04-27T04:32:16.651 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:16.651 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:16.651 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:16.651 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-8.5.0-22.el8.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-8.5.0-22.el8.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:16.652 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:16.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-04-27T04:32:17.528 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:57 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:19.174 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:19.176 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:19.176 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:32:19.176 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:19.176 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:19.176 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 49 k 2024-04-27T04:32:19.176 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-27T04:32:19.177 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout:Install 14 Packages 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.0 M 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 25 M 2024-04-27T04:32:19.178 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:19.679 INFO:teuthology.orchestra.run.smithi138.stdout:(1/14): ceph-mgr-rook-18.2.2-1258.g7d2137f0.el8 277 kB/s | 49 kB 00:00 2024-04-27T04:32:19.729 INFO:teuthology.orchestra.run.smithi138.stdout:(2/14): python3-jsonpointer-1.10-11.el8.noarch. 88 kB/s | 20 kB 00:00 2024-04-27T04:32:19.746 INFO:teuthology.orchestra.run.smithi138.stdout:(3/14): python3-jsonpatch-1.21-2.el8.noarch.rpm 109 kB/s | 27 kB 00:00 2024-04-27T04:32:19.821 INFO:teuthology.orchestra.run.smithi138.stdout:(4/14): python3-jwt-1.6.1-2.el8.noarch.rpm 566 kB/s | 43 kB 00:00 2024-04-27T04:32:19.905 INFO:teuthology.orchestra.run.smithi138.stdout:(5/14): python3-pyasn1-modules-0.3.7-6.el8.noar 625 kB/s | 110 kB 00:00 2024-04-27T04:32:19.930 INFO:teuthology.orchestra.run.smithi138.stdout:(6/14): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.4 MB/s | 155 kB 00:00 2024-04-27T04:32:19.956 INFO:teuthology.orchestra.run.smithi138.stdout:(7/14): python3-requests-oauthlib-1.0.0-1.el8.n 843 kB/s | 43 kB 00:00 2024-04-27T04:32:20.056 INFO:teuthology.orchestra.run.smithi138.stdout:(8/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 335 kB/s | 126 kB 00:00 2024-04-27T04:32:20.082 INFO:teuthology.orchestra.run.smithi138.stdout:(9/14): python3-certifi-2018.10.15-7.el8.noarch 130 kB/s | 16 kB 00:00 2024-04-27T04:32:20.107 INFO:teuthology.orchestra.run.smithi138.stdout:(10/14): python3-cachetools-3.1.1-4.el8.noarch. 187 kB/s | 33 kB 00:00 2024-04-27T04:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout:(11/14): python3-rsa-4.9-2.el8.noarch.rpm 963 kB/s | 64 kB 00:00 2024-04-27T04:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout:(12/14): python3-google-auth-1.1.1-10.el8.noarc 570 kB/s | 81 kB 00:00 2024-04-27T04:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout:(13/14): python3-websocket-client-0.56.0-5.el8. 1.2 MB/s | 61 kB 00:00 2024-04-27T04:32:20.284 INFO:teuthology.orchestra.run.smithi138.stdout:(14/14): python3-kubernetes-11.0.0-6.el8.noarch 5.9 MB/s | 1.2 MB 00:00 2024-04-27T04:32:20.286 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:20.287 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2024-04-27T04:32:20.352 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:20.368 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:20.368 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:20.456 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:20.459 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:21.049 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-27T04:32:21.507 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/14 2024-04-27T04:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/14 2024-04-27T04:32:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/14 2024-04-27T04:32:22.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/14 2024-04-27T04:32:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/14 2024-04-27T04:32:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/14 2024-04-27T04:32:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 8/14 2024-04-27T04:32:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 9/14 2024-04-27T04:32:23.801 INFO:teuthology.orchestra.run.smithi191.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 1.3 MB/s | 14 MB 00:10 2024-04-27T04:32:23.802 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:23.802 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.9 MB/s | 36 MB 00:12 2024-04-27T04:32:24.036 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:32:24.054 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:32:24.054 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:32:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 10/14 2024-04-27T04:32:24.452 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:32:24.455 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:32:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 11/14 2024-04-27T04:32:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 12/14 2024-04-27T04:32:24.876 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 13/14 2024-04-27T04:32:25.037 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:32:25.115 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 14/14 2024-04-27T04:32:25.179 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.5.0-22.el8.x86_64 1/8 2024-04-27T04:32:25.319 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.5.0-22.el8.x86_64 1/8 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 14/14 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 1/14 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/14 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/14 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/14 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/14 2024-04-27T04:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 7/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 8/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 9/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 11/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 12/14 2024-04-27T04:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 13/14 2024-04-27T04:32:25.511 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.5.0-22.el8.x86_64 2/8 2024-04-27T04:32:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 14/14 2024-04-27T04:32:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:25.849 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-27T04:32:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-27T04:32:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:25.851 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:25.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T04:32:26.090 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.5.0-22.el8.x86_64 2/8 2024-04-27T04:32:26.302 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-27T04:32:26.714 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:06 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:26.901 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-27T04:32:27.037 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-27T04:32:27.761 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-27T04:32:28.063 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-27T04:32:28.350 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:28.351 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:32:28.351 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:28.351 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 138 k 2024-04-27T04:32:28.351 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout:Install 5 Packages 2024-04-27T04:32:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:28.353 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-27T04:32:28.353 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 6.0 M 2024-04-27T04:32:28.353 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-04-27T04:32:28.353 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:28.887 INFO:teuthology.orchestra.run.smithi138.stdout:(1/5): ceph-mgr-cephadm-18.2.2-1258.g7d2137f0.e 478 kB/s | 138 kB 00:00 2024-04-27T04:32:29.270 INFO:teuthology.orchestra.run.smithi138.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 860 kB/s | 578 kB 00:00 2024-04-27T04:32:29.688 INFO:teuthology.orchestra.run.smithi138.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 144 kB/s | 60 kB 00:00 2024-04-27T04:32:29.830 INFO:teuthology.orchestra.run.smithi138.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 505 kB/s | 476 kB 00:00 2024-04-27T04:32:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-27T04:32:30.951 INFO:teuthology.orchestra.run.smithi138.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 2.0 MB/s | 4.8 MB 00:02 2024-04-27T04:32:30.953 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:30.954 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.3 MB/s | 6.0 MB 00:02 2024-04-27T04:32:31.009 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:31.021 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:31.021 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:31.193 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:31.196 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:31.381 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-27T04:32:31.626 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f 8/8 2024-04-27T04:32:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f 8/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f 1/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-27T04:32:31.927 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.5.0-22.el8.x86_64 7/8 2024-04-27T04:32:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.5.0-22.el8.x86_64 8/8 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.5.0-22.el8.x86_64 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.5.0-22.el8.x86_64 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-27T04:32:32.350 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-27T04:32:32.351 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:32.351 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:32.471 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-rook 2024-04-27T04:32:33.139 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-27T04:32:33.218 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:01 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:33.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-27T04:32:33.822 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-27T04:32:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 5/5 2024-04-27T04:32:34.319 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 5/5 2024-04-27T04:32:34.319 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/5 2024-04-27T04:32:34.320 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-27T04:32:34.320 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-27T04:32:34.320 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-27T04:32:34.725 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-27T04:32:34.725 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:34.726 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:34.840 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:34.842 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:34.842 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:32:34.842 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:34.842 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:32:34.845 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 49 k 2024-04-27T04:32:34.845 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:32:34.845 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:34.846 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:32:34.847 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:34.847 INFO:teuthology.orchestra.run.smithi191.stdout:Install 14 Packages 2024-04-27T04:32:34.847 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:34.847 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 2.0 M 2024-04-27T04:32:34.847 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 25 M 2024-04-27T04:32:34.847 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:32:34.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-04-27T04:32:35.424 INFO:teuthology.orchestra.run.smithi191.stdout:(1/14): ceph-mgr-rook-18.2.2-1258.g7d2137f0.el8 265 kB/s | 49 kB 00:00 2024-04-27T04:32:35.481 INFO:teuthology.orchestra.run.smithi191.stdout:(2/14): python3-jsonpointer-1.10-11.el8.noarch. 82 kB/s | 20 kB 00:00 2024-04-27T04:32:35.506 INFO:teuthology.orchestra.run.smithi191.stdout:(3/14): python3-jsonpatch-1.21-2.el8.noarch.rpm 99 kB/s | 27 kB 00:00 2024-04-27T04:32:35.612 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:15 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:35.640 INFO:teuthology.orchestra.run.smithi191.stdout:(4/14): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2024-04-27T04:32:35.656 INFO:teuthology.orchestra.run.smithi191.stdout:(5/14): python3-pyasn1-modules-0.3.7-6.el8.noar 626 kB/s | 110 kB 00:00 2024-04-27T04:32:35.741 INFO:teuthology.orchestra.run.smithi191.stdout:(6/14): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.5 MB/s | 155 kB 00:00 2024-04-27T04:32:35.766 INFO:teuthology.orchestra.run.smithi191.stdout:(7/14): python3-requests-oauthlib-1.0.0-1.el8.n 392 kB/s | 43 kB 00:00 2024-04-27T04:32:35.816 INFO:teuthology.orchestra.run.smithi191.stdout:(8/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 322 kB/s | 126 kB 00:00 2024-04-27T04:32:35.933 INFO:teuthology.orchestra.run.smithi191.stdout:(9/14): python3-certifi-2018.10.15-7.el8.noarch 98 kB/s | 16 kB 00:00 2024-04-27T04:32:35.967 INFO:teuthology.orchestra.run.smithi191.stdout:(10/14): python3-cachetools-3.1.1-4.el8.noarch. 146 kB/s | 33 kB 00:00 2024-04-27T04:32:36.025 INFO:teuthology.orchestra.run.smithi191.stdout:(11/14): python3-google-auth-1.1.1-10.el8.noarc 407 kB/s | 81 kB 00:00 2024-04-27T04:32:36.059 INFO:teuthology.orchestra.run.smithi191.stdout:(12/14): python3-rsa-4.9-2.el8.noarch.rpm 699 kB/s | 64 kB 00:00 2024-04-27T04:32:36.092 INFO:teuthology.orchestra.run.smithi191.stdout:(13/14): python3-websocket-client-0.56.0-5.el8. 920 kB/s | 61 kB 00:00 2024-04-27T04:32:36.151 INFO:teuthology.orchestra.run.smithi191.stdout:(14/14): python3-kubernetes-11.0.0-6.el8.noarch 5.4 MB/s | 1.2 MB 00:00 2024-04-27T04:32:36.154 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:36.154 INFO:teuthology.orchestra.run.smithi191.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-04-27T04:32:36.220 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:32:36.236 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:32:36.236 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:32:36.325 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:32:36.328 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:32:36.926 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:32:37.211 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-27T04:32:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 859 k 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-27T04:32:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 942 k 2024-04-27T04:32:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 3.0 M 2024-04-27T04:32:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:37.383 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/14 2024-04-27T04:32:37.549 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 365 kB/s | 83 kB 00:00 2024-04-27T04:32:37.574 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-fuse-18.2.2-1258.g7d2137f0.el8.x86_ 3.3 MB/s | 859 kB 00:00 2024-04-27T04:32:37.574 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:37.574 INFO:teuthology.orchestra.run.smithi138.stdout:Total 3.0 MB/s | 942 kB 00:00 2024-04-27T04:32:37.584 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:37.605 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:37.606 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:37.652 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/14 2024-04-27T04:32:37.692 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:37.695 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:37.870 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:37.890 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/14 2024-04-27T04:32:38.131 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/14 2024-04-27T04:32:38.154 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-27T04:32:38.371 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T04:32:38.397 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/14 2024-04-27T04:32:38.686 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/14 2024-04-27T04:32:38.798 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T04:32:38.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/2 2024-04-27T04:32:38.948 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 8/14 2024-04-27T04:32:39.205 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 9/14 2024-04-27T04:32:39.215 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-27T04:32:39.215 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:39.215 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:39.215 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-27T04:32:39.215 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:39.215 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:39.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-volume 2024-04-27T04:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:20 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:40.290 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 10/14 2024-04-27T04:32:40.565 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 11/14 2024-04-27T04:32:41.068 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 12/14 2024-04-27T04:32:41.827 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 13/14 2024-04-27T04:32:41.871 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:41.872 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:41.872 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T04:32:41.872 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:41.872 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:41.872 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 263 k 2024-04-27T04:32:41.872 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 16.8-9.el8 powertools 48 k 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:41.873 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 311 k 2024-04-27T04:32:41.874 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 1.4 M 2024-04-27T04:32:41.874 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:42.043 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 14/14 2024-04-27T04:32:42.349 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 14/14 2024-04-27T04:32:42.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 1/14 2024-04-27T04:32:42.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 7/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 8/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 9/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 11/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 12/14 2024-04-27T04:32:42.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 13/14 2024-04-27T04:32:42.634 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): ceph-volume-18.2.2-1258.g7d2137f0.el8.no 470 kB/s | 263 kB 00:00 2024-04-27T04:32:42.849 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 14/14 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-27T04:32:42.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-27T04:32:42.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-27T04:32:42.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-27T04:32:42.851 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:42.851 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:42.876 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): python3-packaging-16.8-9.el8.noarch.rpm 60 kB/s | 48 kB 00:00 2024-04-27T04:32:42.877 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:42.878 INFO:teuthology.orchestra.run.smithi138.stdout:Total 310 kB/s | 311 kB 00:01 2024-04-27T04:32:42.890 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:42.902 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:42.903 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:42.969 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T04:32:42.974 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:42.977 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:43.587 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:43.719 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:11 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:43.944 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-27T04:32:44.204 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T04:32:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T04:32:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 1/2 2024-04-27T04:32:45.017 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-27T04:32:45.017 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:45.017 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:45.017 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:45.017 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-27T04:32:45.018 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:45.018 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:45.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-04-27T04:32:45.379 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:45.422 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:32:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 138 k 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:Install 5 Packages 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 6.0 M 2024-04-27T04:32:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 26 M 2024-04-27T04:32:45.424 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:32:45.993 INFO:teuthology.orchestra.run.smithi191.stdout:(1/5): ceph-mgr-cephadm-18.2.2-1258.g7d2137f0.e 462 kB/s | 138 kB 00:00 2024-04-27T04:32:46.056 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:26 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:46.301 INFO:teuthology.orchestra.run.smithi191.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 952 kB/s | 578 kB 00:00 2024-04-27T04:32:46.451 INFO:teuthology.orchestra.run.smithi191.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.0 MB/s | 476 kB 00:00 2024-04-27T04:32:46.594 INFO:teuthology.orchestra.run.smithi191.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 206 kB/s | 60 kB 00:00 2024-04-27T04:32:47.477 INFO:teuthology.orchestra.run.smithi191.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 2.7 MB/s | 4.8 MB 00:01 2024-04-27T04:32:47.479 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:47.480 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.9 MB/s | 6.0 MB 00:02 2024-04-27T04:32:47.533 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:32:47.545 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:32:47.545 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:32:47.658 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:32:47.661 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:32:47.711 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 139 k 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:32:47.712 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:47.713 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 139 k 2024-04-27T04:32:47.713 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 513 k 2024-04-27T04:32:47.713 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:47.900 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-18.2.2-1258.g7d2137f0.el8.x86_64 745 kB/s | 139 kB 00:00 2024-04-27T04:32:47.900 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:47.901 INFO:teuthology.orchestra.run.smithi138.stdout:Total 739 kB/s | 139 kB 00:00 2024-04-27T04:32:47.901 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:47.906 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:47.909 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:47.955 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:47.958 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:47.961 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:32:48.064 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:32:48.278 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-27T04:32:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:32:49.218 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-27T04:32:49.237 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:32:49.237 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:49.237 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:49.237 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:32:49.237 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:49.237 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:49.602 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-27T04:32:49.942 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-27T04:32:49.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-04-27T04:32:50.173 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 5/5 2024-04-27T04:32:50.465 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 5/5 2024-04-27T04:32:50.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/5 2024-04-27T04:32:50.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-27T04:32:50.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-27T04:32:50.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-27T04:32:50.715 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:30 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:50.864 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-27T04:32:50.864 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:50.864 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:32:50.864 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:32:50.865 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-27T04:32:50.865 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-27T04:32:50.865 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-04-27T04:32:50.865 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-27T04:32:50.865 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:50.865 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:50.979 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-fuse 2024-04-27T04:32:51.734 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:19 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:52.328 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:32:52.375 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:52.376 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:32:52.377 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:52.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-04-27T04:32:53.256 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:33 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:53.391 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 859 k 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-27T04:32:53.392 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:53.393 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:32:53.393 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:53.393 INFO:teuthology.orchestra.run.smithi191.stdout:Install 2 Packages 2024-04-27T04:32:53.393 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:53.393 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 942 k 2024-04-27T04:32:53.394 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 3.0 M 2024-04-27T04:32:53.394 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:32:53.551 INFO:teuthology.orchestra.run.smithi191.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 791 kB/s | 83 kB 00:00 2024-04-27T04:32:53.734 INFO:teuthology.orchestra.run.smithi191.stdout:(2/2): ceph-fuse-18.2.2-1258.g7d2137f0.el8.x86_ 2.9 MB/s | 859 kB 00:00 2024-04-27T04:32:53.735 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:53.735 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.7 MB/s | 942 kB 00:00 2024-04-27T04:32:53.745 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:32:53.766 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:32:53.766 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:32:53.851 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:32:53.854 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:32:54.073 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:32:54.391 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-27T04:32:54.597 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T04:32:54.907 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:54.907 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:54.907 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 31 k 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:32:54.908 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:54.909 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 31 k 2024-04-27T04:32:54.909 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 139 k 2024-04-27T04:32:54.909 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:32:54.981 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T04:32:54.982 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/2 2024-04-27T04:32:55.144 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-18.2.2-1258.g7d2137f0.el8.x86_6 131 kB/s | 31 kB 00:00 2024-04-27T04:32:55.144 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:55.145 INFO:teuthology.orchestra.run.smithi138.stdout:Total 131 kB/s | 31 kB 00:00 2024-04-27T04:32:55.145 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:32:55.149 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:32:55.149 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:32:55.170 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:32:55.173 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:32:55.231 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:32:55.356 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-27T04:32:55.356 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:55.356 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:32:55.356 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-27T04:32:55.356 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:55.356 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:32:55.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:32:55.619 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-volume 2024-04-27T04:32:55.692 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:32:56.063 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:32:56.063 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:56.063 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:32:56.063 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:32:56.063 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:32:56.063 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:56.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-04-27T04:32:56.383 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:24 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:32:56.941 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:36 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:58.051 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:32:58.052 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:58.052 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-04-27T04:32:58.052 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:58.052 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:32:58.052 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 263 k 2024-04-27T04:32:58.052 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-04-27T04:32:58.053 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 16.8-9.el8 powertools 48 k 2024-04-27T04:32:58.053 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:58.053 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:32:58.053 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:32:58.053 INFO:teuthology.orchestra.run.smithi191.stdout:Install 2 Packages 2024-04-27T04:32:58.053 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:32:58.054 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 311 k 2024-04-27T04:32:58.054 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 1.4 M 2024-04-27T04:32:58.054 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:32:58.323 INFO:teuthology.orchestra.run.smithi191.stdout:(1/2): ceph-volume-18.2.2-1258.g7d2137f0.el8.no 1.2 MB/s | 263 kB 00:00 2024-04-27T04:32:58.348 INFO:teuthology.orchestra.run.smithi191.stdout:(2/2): python3-packaging-16.8-9.el8.noarch.rpm 199 kB/s | 48 kB 00:00 2024-04-27T04:32:58.348 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:32:58.349 INFO:teuthology.orchestra.run.smithi191.stdout:Total 1.0 MB/s | 311 kB 00:00 2024-04-27T04:32:58.358 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:32:58.370 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:32:58.370 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:32:58.441 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:32:58.445 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:32:58.544 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:32:58.586 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:32:58.592 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:32:58.593 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:32:58.593 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:32:58.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-04-27T04:32:58.919 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-27T04:32:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T04:32:59.460 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:39 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:32:59.656 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T04:32:59.656 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 1/2 2024-04-27T04:33:00.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-27T04:33:00.068 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:00.068 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:33:00.069 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T04:33:00.069 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-27T04:33:00.069 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:00.069 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:00.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install librados-devel 2024-04-27T04:33:01.066 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:01.095 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:29 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:01.114 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:01.116 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:33:01.116 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:01.244 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-04-27T04:33:02.005 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:42 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:02.755 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:02.755 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repo Size 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 139 k 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 139 k 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 513 k 2024-04-27T04:33:02.756 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:33:02.936 INFO:teuthology.orchestra.run.smithi191.stdout:librados-devel-18.2.2-1258.g7d2137f0.el8.x86_64 774 kB/s | 139 kB 00:00 2024-04-27T04:33:02.936 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:02.936 INFO:teuthology.orchestra.run.smithi191.stdout:Total 768 kB/s | 139 kB 00:00 2024-04-27T04:33:02.937 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:33:02.942 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:33:02.942 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:33:02.990 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:33:02.993 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:33:03.106 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:03.662 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:03.663 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:33:03.663 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:03.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-04-27T04:33:03.797 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:04.540 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:44 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:04.541 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:04.979 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:04.979 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:04.979 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:33:04.979 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:04.979 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:04.979 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:05.218 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install libcephfs2 2024-04-27T04:33:05.985 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:33 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:06.158 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:06.206 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:06.207 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:33:06.207 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:06.312 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-04-27T04:33:07.069 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:47 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:07.597 INFO:teuthology.orchestra.run.smithi191.stdout:Package libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:07.644 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:07.645 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:07.645 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:07.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install libcephfs-devel 2024-04-27T04:33:08.501 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:36 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:08.709 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:08.710 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:33:08.710 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:08.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-04-27T04:33:09.578 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:49 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:10.166 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repo Size 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 31 k 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:10.167 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:33:10.168 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:10.168 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:33:10.168 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:10.168 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 31 k 2024-04-27T04:33:10.168 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 139 k 2024-04-27T04:33:10.168 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:33:10.518 INFO:teuthology.orchestra.run.smithi191.stdout:libcephfs-devel-18.2.2-1258.g7d2137f0.el8.x86_6 88 kB/s | 31 kB 00:00 2024-04-27T04:33:10.519 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:10.519 INFO:teuthology.orchestra.run.smithi191.stdout:Total 88 kB/s | 31 kB 00:00 2024-04-27T04:33:10.519 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:33:10.523 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:33:10.523 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:33:10.555 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:33:10.558 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:33:10.613 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:33:10.843 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:11.089 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:11.191 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:11.239 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:11.241 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T04:33:11.241 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:11.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-04-27T04:33:11.450 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:11.450 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:11.451 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:33:11.451 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:11.451 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:11.451 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:11.567 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install librados2 2024-04-27T04:33:12.097 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:52 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:12.317 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:40 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:13.735 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 95 k 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 95 k 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 280 k 2024-04-27T04:33:13.736 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:33:13.918 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:13.924 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 507 kB/s | 95 kB 00:00 2024-04-27T04:33:13.924 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:13.925 INFO:teuthology.orchestra.run.smithi138.stdout:Total 504 kB/s | 95 kB 00:00 2024-04-27T04:33:13.925 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:33:13.935 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:33:13.935 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:33:13.966 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:13.967 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:13.967 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:13.985 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:33:13.989 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:33:14.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install librbd1 2024-04-27T04:33:14.097 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:33:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:14.701 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:14.834 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:42 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:15.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:15.082 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:33:15.082 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:15.082 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:15.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-04-27T04:33:16.073 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:56 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:16.440 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:16.488 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:16.489 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:16.489 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:16.594 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-rados 2024-04-27T04:33:17.367 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:45 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:17.706 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:17.706 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T04:33:17.706 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:17.706 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:33:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.5 M 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.5 M 2024-04-27T04:33:17.707 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 16 M 2024-04-27T04:33:17.708 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:33:18.238 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 6.5 MB/s | 3.5 MB 00:00 2024-04-27T04:33:18.239 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:18.239 INFO:teuthology.orchestra.run.smithi138.stdout:Total 6.5 MB/s | 3.5 MB 00:00 2024-04-27T04:33:18.239 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:33:18.251 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:33:18.251 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:33:18.313 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:33:18.317 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:33:18.752 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:33:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:18.966 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:19.014 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:19.015 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:19.015 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:19.127 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-rgw 2024-04-27T04:33:19.368 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:19.782 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:19.782 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:19.782 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:33:19.783 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:19.783 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:19.783 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:19.882 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:47 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:19.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-04-27T04:33:20.739 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:03:00 ago on Sat 27 Apr 2024 04:30:20 AM UTC. 2024-04-27T04:33:21.467 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:21.515 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:21.516 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:21.516 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:21.619 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-cephfs 2024-04-27T04:33:22.376 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:50 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:22.389 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T04:33:22.390 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T04:33:22.390 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:22.390 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T04:33:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 182 k 2024-04-27T04:33:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:22.391 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T04:33:22.391 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T04:33:22.391 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T04:33:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:22.391 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 182 k 2024-04-27T04:33:22.392 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 572 k 2024-04-27T04:33:22.392 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T04:33:22.577 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 983 kB/s | 182 kB 00:00 2024-04-27T04:33:22.577 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:22.578 INFO:teuthology.orchestra.run.smithi138.stdout:Total 976 kB/s | 182 kB 00:00 2024-04-27T04:33:22.578 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T04:33:22.589 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T04:33:22.589 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T04:33:22.639 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T04:33:22.642 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T04:33:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T04:33:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:23.626 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T04:33:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:23.626 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T04:33:23.889 DEBUG:teuthology.parallel:result is None 2024-04-27T04:33:23.983 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:24.030 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:24.032 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:24.032 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:24.139 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-rbd 2024-04-27T04:33:24.889 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:52 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:26.488 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T04:33:26.535 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:26.536 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-04-27T04:33:26.536 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:26.639 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install rbd-fuse 2024-04-27T04:33:27.389 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:55 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:29.042 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:29.043 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 95 k 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 95 k 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 280 k 2024-04-27T04:33:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:33:29.338 INFO:teuthology.orchestra.run.smithi191.stdout:rbd-fuse-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 323 kB/s | 95 kB 00:00 2024-04-27T04:33:29.339 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:29.339 INFO:teuthology.orchestra.run.smithi191.stdout:Total 321 kB/s | 95 kB 00:00 2024-04-27T04:33:29.339 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:33:29.350 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:33:29.350 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:33:29.398 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:33:29.401 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:33:29.503 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:33:29.717 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:29.991 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:30.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:30.372 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:30.372 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:33:30.372 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:30.372 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:30.372 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:30.638 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install rbd-mirror 2024-04-27T04:33:31.403 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:59 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:33.046 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repo Size 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.5 M 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:33:33.047 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:33.048 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 3.5 M 2024-04-27T04:33:33.048 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 16 M 2024-04-27T04:33:33.048 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:33:33.408 INFO:teuthology.orchestra.run.smithi191.stdout:rbd-mirror-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 9.6 MB/s | 3.5 MB 00:00 2024-04-27T04:33:33.409 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:33.410 INFO:teuthology.orchestra.run.smithi191.stdout:Total 9.5 MB/s | 3.5 MB 00:00 2024-04-27T04:33:33.410 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:33:33.421 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:33:33.421 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:33:33.485 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:33:33.488 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:33:33.927 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:34.549 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:34.924 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:34.924 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:34.924 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:33:34.924 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:34.924 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:34.924 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:35.144 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install rbd-nbd 2024-04-27T04:33:35.915 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:03:03 ago on Sat 27 Apr 2024 04:30:32 AM UTC. 2024-04-27T04:33:37.564 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-04-27T04:33:37.564 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:37.564 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-04-27T04:33:37.564 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:37.564 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 182 k 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:37.565 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 182 k 2024-04-27T04:33:37.566 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 572 k 2024-04-27T04:33:37.566 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-04-27T04:33:37.761 INFO:teuthology.orchestra.run.smithi191.stdout:rbd-nbd-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 934 kB/s | 182 kB 00:00 2024-04-27T04:33:37.761 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-04-27T04:33:37.761 INFO:teuthology.orchestra.run.smithi191.stdout:Total 928 kB/s | 182 kB 00:00 2024-04-27T04:33:37.762 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-04-27T04:33:37.773 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-04-27T04:33:37.773 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-04-27T04:33:37.824 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-04-27T04:33:37.827 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-04-27T04:33:37.938 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-04-27T04:33:38.114 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:38.391 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:38.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T04:33:38.771 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:38.771 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-04-27T04:33:38.771 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T04:33:38.771 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:38.771 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-04-27T04:33:39.010 DEBUG:teuthology.parallel:result is None 2024-04-27T04:33:39.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T04:33:39.154 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T04:33:39.181 INFO:teuthology.orchestra.run.smithi138.stdout:18.2.2-1258.g7d2137f0.el8 2024-04-27T04:33:39.181 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1258.g7d2137f0.el8 2024-04-27T04:33:39.182 INFO:teuthology.task.install:The correct ceph version 18.2.2-1258.g7d2137f0 is installed. 2024-04-27T04:33:39.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T04:33:39.421 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T04:33:39.443 INFO:teuthology.orchestra.run.smithi191.stdout:18.2.2-1258.g7d2137f0.el8 2024-04-27T04:33:39.443 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1258.g7d2137f0.el8 2024-04-27T04:33:39.443 INFO:teuthology.task.install:The correct ceph version 18.2.2-1258.g7d2137f0 is installed. 2024-04-27T04:33:39.445 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-27T04:33:39.445 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:39.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T04:33:39.479 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:39.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T04:33:39.510 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-27T04:33:39.511 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:39.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T04:33:39.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T04:33:39.621 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:39.621 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T04:33:39.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T04:33:39.715 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-27T04:33:39.715 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:39.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T04:33:39.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T04:33:39.815 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:39.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T04:33:39.842 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T04:33:39.908 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-27T04:33:39.908 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:39.908 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T04:33:39.937 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T04:33:40.006 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:40.006 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T04:33:40.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T04:33:40.099 INFO:teuthology.run_tasks:Running task ceph... 2024-04-27T04:33:40.195 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-27T04:33:40.195 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-04-27T04:33:40.197 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2024-04-27T04:33:40.223 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-27T04:33:40.224 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T04:33:40.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T04:33:40.292 INFO:tasks.ceph:Creating extra log directories... 2024-04-27T04:33:40.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T04:33:40.334 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T04:33:40.368 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-27T04:33:40.368 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-27T04:33:40.368 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699', 'branch': 'reef', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7675699', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'wait-for-scrub': False}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6'}}, 'rgw': {'ec-data-pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'frontend': 'beast'}, 's3tests': {'slow_backend': True}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '5403a02127f27ec6c3309ef057522ab70cdeb9d2'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 4225, 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'rgw', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '5403a02127f27ec6c3309ef057522ab70cdeb9d2', 'targets': {'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJUUxDBMHKa3h4hU+1Te0s5UfW3KtoI2wf2oGIlmmCaR+z0Qsf/yQ7S7ElSUVqaJSQGhcM1GrSvfbDENSd4FMc=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8CnoVCmRVutcU2qpT9plruGZg54nSzkONz9hWrprdl9G3Vmwr3TY3xjGxYQThhDKrkdssFmKUcOH4MmyDu2fg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'wait-for-scrub': False, 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_22:40:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-27T04:33:40.369 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T04:33:40.401 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T04:33:40.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T04:33:40.459 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T04:33:40.488 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:40.488 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T04:33:40.545 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T04:33:40.545 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 164453 Links: 1 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 04:33:19.117042123 +0000 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 04:27:51.301938599 +0000 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 04:27:51.301938599 +0000 2024-04-27T04:33:40.604 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-27T04:33:40.604 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T04:33:40.670 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T04:33:40.670 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T04:33:40.670 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000232694 s, 2.2 MB/s 2024-04-27T04:33:40.671 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T04:33:40.730 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 165827 Links: 1 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 04:33:19.117042123 +0000 2024-04-27T04:33:40.788 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 04:27:51.631932125 +0000 2024-04-27T04:33:40.789 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 04:27:51.631932125 +0000 2024-04-27T04:33:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-27T04:33:40.789 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T04:33:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T04:33:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T04:33:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000258862 s, 2.0 MB/s 2024-04-27T04:33:40.855 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T04:33:40.913 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-27T04:33:40.970 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 164523 Links: 1 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 04:33:19.117042123 +0000 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 04:27:51.967925533 +0000 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 04:27:51.967925533 +0000 2024-04-27T04:33:40.971 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-27T04:33:40.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T04:33:41.036 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T04:33:41.036 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T04:33:41.036 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000180078 s, 2.8 MB/s 2024-04-27T04:33:41.037 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T04:33:41.096 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 166701 Links: 1 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 04:33:19.117042123 +0000 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 04:27:52.307918864 +0000 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 04:27:52.307918864 +0000 2024-04-27T04:33:41.154 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-27T04:33:41.155 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T04:33:41.221 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T04:33:41.221 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T04:33:41.221 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000231997 s, 2.2 MB/s 2024-04-27T04:33:41.221 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T04:33:41.279 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-27T04:33:41.280 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:41.280 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T04:33:41.302 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T04:33:41.302 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-04-27T04:33:41.362 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 171248 Links: 1 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-04-27 04:33:34.297196331 +0000 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-04-27 04:27:55.286351397 +0000 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-04-27 04:27:55.286351397 +0000 2024-04-27T04:33:41.363 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-04-27T04:33:41.363 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T04:33:41.432 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-04-27T04:33:41.433 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-04-27T04:33:41.433 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000237984 s, 2.2 MB/s 2024-04-27T04:33:41.434 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T04:33:41.495 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 170432 Links: 1 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-04-27 04:33:34.297196331 +0000 2024-04-27T04:33:41.556 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-04-27 04:27:55.657345126 +0000 2024-04-27T04:33:41.557 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-04-27 04:27:55.657345126 +0000 2024-04-27T04:33:41.557 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-04-27T04:33:41.557 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T04:33:41.623 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-04-27T04:33:41.623 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-04-27T04:33:41.623 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000221554 s, 2.3 MB/s 2024-04-27T04:33:41.625 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T04:33:41.684 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-04-27T04:33:41.743 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T04:33:41.743 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:41.743 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 165474 Links: 1 2024-04-27T04:33:41.743 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:41.743 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:41.744 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-04-27 04:33:34.298196314 +0000 2024-04-27T04:33:41.744 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-04-27 04:27:55.992339464 +0000 2024-04-27T04:33:41.744 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-04-27 04:27:55.992339464 +0000 2024-04-27T04:33:41.744 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-04-27T04:33:41.744 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T04:33:41.809 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-04-27T04:33:41.810 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-04-27T04:33:41.810 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000182837 s, 2.8 MB/s 2024-04-27T04:33:41.811 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T04:33:41.869 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-04-27T04:33:41.927 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T04:33:41.927 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T04:33:41.927 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 168604 Links: 1 2024-04-27T04:33:41.928 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T04:33:41.928 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T04:33:41.928 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-04-27 04:33:34.298196314 +0000 2024-04-27T04:33:41.928 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-04-27 04:27:56.323333870 +0000 2024-04-27T04:33:41.928 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-04-27 04:27:56.323333870 +0000 2024-04-27T04:33:41.928 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-04-27T04:33:41.928 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T04:33:41.993 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-04-27T04:33:41.993 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-04-27T04:33:41.993 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000230676 s, 2.2 MB/s 2024-04-27T04:33:41.994 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T04:33:42.053 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T04:33:42.054 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-27T04:33:42.054 INFO:tasks.ceph:Generating config... 2024-04-27T04:33:42.055 INFO:tasks.ceph:[client] debug ms = 1 2024-04-27T04:33:42.055 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-04-27T04:33:42.055 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-27T04:33:42.055 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-27T04:33:42.055 INFO:tasks.ceph:[client] setuser = ceph 2024-04-27T04:33:42.055 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-27T04:33:42.055 INFO:tasks.ceph:[osd] debug objclass = 20 2024-04-27T04:33:42.055 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-27T04:33:42.055 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-27T04:33:42.055 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-27T04:33:42.056 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-27T04:33:42.056 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-27T04:33:42.056 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-27T04:33:42.056 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-27T04:33:42.057 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-27T04:33:42.057 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-27T04:33:42.057 INFO:tasks.ceph:Setting up mon.a... 2024-04-27T04:33:42.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-27T04:33:42.126 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.keyring 2024-04-27T04:33:42.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-27T04:33:42.177 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T04:33:42.205 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.138')] 2024-04-27T04:33:42.205 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.138'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-04-27T04:33:42.206 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:42.206 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T04:33:42.264 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.138 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:42.347 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:33:42.348 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:33:42.349 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:42.350 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: generated fsid 512e09fb-564f-4fb6-aa0b-f541dbeb017d 2024-04-27T04:33:42.350 INFO:teuthology.orchestra.run.smithi138.stdout:setting min_mon_release = pacific 2024-04-27T04:33:42.350 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 0 2024-04-27T04:33:42.350 INFO:teuthology.orchestra.run.smithi138.stdout:fsid 512e09fb-564f-4fb6-aa0b-f541dbeb017d 2024-04-27T04:33:42.350 INFO:teuthology.orchestra.run.smithi138.stdout:last_changed 2024-04-27T04:33:42.347968+0000 2024-04-27T04:33:42.351 INFO:teuthology.orchestra.run.smithi138.stdout:created 2024-04-27T04:33:42.347968+0000 2024-04-27T04:33:42.351 INFO:teuthology.orchestra.run.smithi138.stdout:min_mon_release 16 (pacific) 2024-04-27T04:33:42.351 INFO:teuthology.orchestra.run.smithi138.stdout:election_strategy: 1 2024-04-27T04:33:42.351 INFO:teuthology.orchestra.run.smithi138.stdout:0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-04-27T04:33:42.351 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-27T04:33:42.352 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T04:33:42.408 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 512e09fb-564f-4fb6-aa0b-f541dbeb017d... 2024-04-27T04:33:42.409 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T04:33:42.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T04:33:42.492 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-04-27T04:33:42.493 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-04-27T04:33:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T04:33:42.495 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.496 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = 172.21.15.138 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 512e09fb-564f-4fb6-aa0b-f541dbeb017d 2024-04-27T04:33:42.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-04-27T04:33:42.498 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-04-27T04:33:42.499 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: debug objclass = 20 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: bdev async discard = True 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: bdev enable discard = True 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore allocator = bitmap 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 1/20 2024-04-27T04:33:42.500 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 1/20 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 4/10 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-27T04:33:42.501 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-27T04:33:42.502 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T04:33:42.503 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: rgw gc obj min wait = 15 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: debug rgw = 20 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: setgroup = ceph 2024-04-27T04:33:42.504 INFO:teuthology.orchestra.run.smithi138.stdout: setuser = ceph 2024-04-27T04:33:42.505 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-04-27T04:33:42.505 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2024-04-27T04:33:42.505 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2024-04-27T04:33:42.505 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T04:33:42.505 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2024-04-27T04:33:42.505 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.506 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T04:33:42.507 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2024-04-27T04:33:42.508 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = 172.21.15.138 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 512e09fb-564f-4fb6-aa0b-f541dbeb017d 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.509 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2024-04-27T04:33:42.510 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-04-27T04:33:42.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug objclass = 20 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2024-04-27T04:33:42.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-04-27T04:33:42.513 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-04-27T04:33:42.514 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2024-04-27T04:33:42.515 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: rgw gc obj min wait = 15 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: debug rgw = 20 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: setgroup = ceph 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout: setuser = ceph 2024-04-27T04:33:42.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2024-04-27T04:33:42.518 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-27T04:33:42.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-27T04:33:42.614 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-27T04:33:42.615 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:42.615 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-27T04:33:42.632 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:42.632 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-27T04:33:42.689 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T04:33:42.689 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:42.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T04:33:42.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T04:33:42.765 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:42.765 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:42.823 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:33:42.823 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:42.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T04:33:42.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T04:33:42.860 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:42.861 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:42.919 INFO:tasks.ceph:Setting up mon nodes... 2024-04-27T04:33:42.919 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-27T04:33:42.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T04:33:43.036 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T04:33:43.039 INFO:tasks.ceph:Setting up mds nodes... 2024-04-27T04:33:43.039 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-27T04:33:43.040 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-27T04:33:43.084 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-27T04:33:43.095 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-27T04:33:43.096 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-27T04:33:43.096 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-27T04:33:43.123 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-27T04:33:43.123 INFO:tasks.ceph:role: osd.0 2024-04-27T04:33:43.123 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T04:33:43.124 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T04:33:43.197 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T04:33:43.198 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T04:33:43.628 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-27T04:33:43.630 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-27T04:33:43.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-27T04:33:43.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-27T04:33:43.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:43.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:43.924+0000 7f6e36ef57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-27T04:33:43.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:43.927+0000 7f6e36ef57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-27T04:33:43.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:43.927+0000 7f6e36ef57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T04:33:43.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:43.927+0000 7f6e36ef57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T04:33:43.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:43.927+0000 7f6e36ef57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T04:33:43.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:43.934+0000 7f6e36ef57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-27T04:33:46.787 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T04:33:46.820 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-27T04:33:46.854 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T04:33:46.855 INFO:tasks.ceph:role: osd.1 2024-04-27T04:33:46.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:33:46.855 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T04:33:46.934 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T04:33:46.934 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T04:33:46.934 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T04:33:46.934 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T04:33:46.935 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T04:33:46.935 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-04-27T04:33:46.935 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T04:33:46.935 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T04:33:46.935 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T04:33:46.935 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T04:33:47.369 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-04-27T04:33:47.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-04-27T04:33:47.371 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-27T04:33:47.536 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-27T04:33:47.572 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-27T04:33:47.641 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T04:33:47.641 INFO:tasks.ceph:role: osd.2 2024-04-27T04:33:47.642 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:33:47.642 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T04:33:47.717 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T04:33:47.717 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T04:33:47.718 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T04:33:48.141 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-04-27T04:33:48.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-04-27T04:33:48.143 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-04-27T04:33:48.248 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-27T04:33:48.280 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-27T04:33:48.348 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T04:33:48.348 INFO:tasks.ceph:role: osd.3 2024-04-27T04:33:48.348 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:33:48.348 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T04:33:48.423 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T04:33:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T04:33:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T04:33:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T04:33:48.423 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T04:33:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-04-27T04:33:48.424 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T04:33:48.424 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T04:33:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T04:33:48.424 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T04:33:48.868 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-04-27T04:33:48.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-04-27T04:33:48.870 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-04-27T04:33:49.021 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-27T04:33:49.055 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:49.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:49.160+0000 7f409b1477c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-27T04:33:49.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:49.162+0000 7f409b1477c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-27T04:33:49.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:49.162+0000 7f409b1477c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T04:33:49.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:49.162+0000 7f409b1477c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T04:33:49.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:49.162+0000 7f409b1477c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T04:33:49.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:49.171+0000 7f409b1477c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-27T04:33:52.027 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T04:33:52.058 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:52.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:52.149+0000 7fd3b9a347c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-27T04:33:52.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:52.149+0000 7fd3b9a347c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-27T04:33:52.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:52.149+0000 7fd3b9a347c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T04:33:52.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:52.149+0000 7fd3b9a347c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T04:33:52.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:52.149+0000 7fd3b9a347c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T04:33:52.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:52.150+0000 7fd3b9a347c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-27T04:33:55.006 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T04:33:55.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:55.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:55.129+0000 7fbb984417c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-27T04:33:55.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:55.130+0000 7fbb984417c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-27T04:33:55.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:55.130+0000 7fbb984417c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T04:33:55.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:55.130+0000 7fbb984417c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T04:33:55.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:55.130+0000 7fbb984417c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T04:33:55.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:55.130+0000 7fbb984417c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-27T04:33:57.975 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T04:33:58.008 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-27T04:33:58.008 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:58.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-27T04:33:58.041 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:58.041 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-27T04:33:58.078 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:58.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-27T04:33:58.147 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:58.147 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-27T04:33:58.215 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:58.215 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-27T04:33:58.283 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:58.283 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-27T04:33:58.341 INFO:tasks.ceph:Adding keys to all mons... 2024-04-27T04:33:58.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T04:33:58.372 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-04-27T04:33:58.372 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAngCxmyXYxNxAA4d11jjf2HItftXcHIEanPw== 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAngCxmY4MvAhAAX+6FxsUF+DsVufH5qaPuiA== 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAtgCxmD7eqCRAAc7IAaDu2Kthiy3HFkpLjRA== 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAwgCxmR/r3CBAAPrNXIFFGjiW8TSYz0tdxgQ== 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAzgCxmoITMBxAAil0plyKD6ov/yCygSeEkQA== 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-04-27T04:33:58.373 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAngCxmvMb9BBAA6tlDK84gMHtVfCGEkVqFTA== 2024-04-27T04:33:58.374 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T04:33:58.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T04:33:58.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T04:33:58.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T04:33:58.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T04:33:58.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T04:33:58.747 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-27T04:33:58.747 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-27T04:33:58.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T04:33:58.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-27T04:33:58.897 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T04:33:58.955 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-27T04:33:58.955 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-27T04:33:58.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-27T04:33:58.997 INFO:tasks.ceph.mon.a:Started 2024-04-27T04:33:58.998 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-27T04:33:58.998 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-27T04:33:58.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-27T04:33:59.000 INFO:tasks.ceph.mgr.x:Started 2024-04-27T04:33:59.000 INFO:tasks.ceph:Setting crush tunables to default 2024-04-27T04:33:59.001 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd crush tunables default 2024-04-27T04:33:59.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.121+0000 7f733174f700 1 Processor -- start 2024-04-27T04:33:59.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.121+0000 7f733174f700 1 -- start start 2024-04-27T04:33:59.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f733174f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f733174f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c072d20 con 0x7f732c12e960 2024-04-27T04:33:59.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f733174f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c097160 con 0x7f732c12e040 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59022/0 (socket says 172.21.15.138:59022) 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 -- 172.21.15.138:0/1661411601 learned_addr learned my addr 172.21.15.138:0/1661411601 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732a7fc700 1 --1- 172.21.15.138:0/1661411601 >> v1:172.21.15.138:6789/0 conn(0x7f732c12e960 0x7f732c095180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59776/0 (socket says 172.21.15.138:59776) 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 -- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 msgr2=0x7f732c12e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 -- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 msgr2=0x7f732c12e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 --2- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T04:33:59.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.122+0000 7f732affd700 1 --2- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.126+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257511258 0 0) 0x7f732c072d20 con 0x7f732c12e960 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.126+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73140035b0 con 0x7f732c12e960 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2844195059 0 0) 0x7f73140035b0 con 0x7f732c12e960 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 msgr2=0x7f732c12e420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 --2- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c097d80 con 0x7f732c12e960 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 2155127222 0 0) 0x7f73200015a0 con 0x7f732c12e960 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 >> v1:172.21.15.138:6789/0 conn(0x7f732c12e960 legacy=0x7f732c095180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 --2- 172.21.15.138:0/1661411601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f7314003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7314004550 con 0x7f7314003a00 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f732affd700 1 --2- 172.21.15.138:0/1661411601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f7314003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f732affd700 1 -- 172.21.15.138:0/1661411601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140035b0 con 0x7f7314003a00 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f732affd700 1 --2- 172.21.15.138:0/1661411601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f7314003e00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f731c009890 tx=0x7f7314005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b91698a013fa38 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f731c00c070 con 0x7f7314003a00 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c004170 con 0x7f7314003a00 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.127+0000 7f7329ffb700 1 -- 172.21.15.138:0/1661411601 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f731c004450 con 0x7f7314003a00 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 -- 172.21.15.138:0/1661411601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 msgr2=0x7f7314003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 --2- 172.21.15.138:0/1661411601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f7314003e00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f731c009890 tx=0x7f7314005880 comp rx=0 tx=0).stop 2024-04-27T04:33:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 -- 172.21.15.138:0/1661411601 shutdown_connections 2024-04-27T04:33:59.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 --2- 172.21.15.138:0/1661411601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f7314003e00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 --2- 172.21.15.138:0/1661411601 >> v2:172.21.15.138:3300/0 conn(0x7f732c12e040 0x7f732c12e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 -- 172.21.15.138:0/1661411601 >> 172.21.15.138:0/1661411601 conn(0x7f732c09ded0 msgr2=0x7f732c09e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:33:59.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 -- 172.21.15.138:0/1661411601 shutdown_connections 2024-04-27T04:33:59.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.128+0000 7f733174f700 1 -- 172.21.15.138:0/1661411601 wait complete. 2024-04-27T04:33:59.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.129+0000 7f733174f700 1 Processor -- start 2024-04-27T04:33:59.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.129+0000 7f733174f700 1 -- start start 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.129+0000 7f733174f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f732c1c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.129+0000 7f733174f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0726b0 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.129+0000 7f732affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f732c1c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f732affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f732c1c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59048/0 (socket says 172.21.15.138:59048) 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f732affd700 1 -- 172.21.15.138:0/1204653171 learned_addr learned my addr 172.21.15.138:0/1204653171 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f732affd700 1 -- 172.21.15.138:0/1204653171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c123e60 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f732affd700 1 --2- 172.21.15.138:0/1204653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f732c1c1e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f731c016040 tx=0x7f731c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f731c003fc0 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c01c470 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c093be0 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f731c01c730 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c123770 con 0x7f7314003a00 2024-04-27T04:33:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f731c01cb90 con 0x7f7314003a00 2024-04-27T04:33:59.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.130+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f731c030440 con 0x7f7314003a00 2024-04-27T04:33:59.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.131+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c123ff0 con 0x7f7314003a00 2024-04-27T04:33:59.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.133+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f731c02b070 con 0x7f7314003a00 2024-04-27T04:33:59.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.206+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f732c123c50 con 0x7f7314003a00 2024-04-27T04:33:59.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.208+0000 7f73137fe700 1 -- 172.21.15.138:0/1204653171 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f731c017030 con 0x7f7314003a00 2024-04-27T04:33:59.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 msgr2=0x7f732c1c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 --2- 172.21.15.138:0/1204653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f732c1c1e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f731c016040 tx=0x7f731c0047c0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 shutdown_connections 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 --2- 172.21.15.138:0/1204653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7314003a00 0x7f732c1c1e20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 >> 172.21.15.138:0/1204653171 conn(0x7f732c09ded0 msgr2=0x7f732c06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 shutdown_connections 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:33:59.209+0000 7f733174f700 1 -- 172.21.15.138:0/1204653171 wait complete. 2024-04-27T04:33:59.211 INFO:teuthology.orchestra.run.smithi138.stderr:adjusted tunables profile to default 2024-04-27T04:33:59.222 INFO:tasks.ceph:check_enable_crimson: False 2024-04-27T04:33:59.222 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-27T04:33:59.222 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T04:33:59.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-27T04:33:59.247 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:33:59.246+0000 7f77976a0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T04:33:59.249 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:59.249 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-27T04:33:59.275 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:59.275 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-27T04:33:59.343 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-04-27T04:33:59.343 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-27T04:33:59.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 74a1ee1e-2f1c-4eaa-860a-ad1e52733bde 0 2024-04-27T04:33:59.429 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:33:59.428+0000 7f77976a0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.549+0000 7f0eb21ea700 1 Processor -- start 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eb21ea700 1 -- start start 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eb21ea700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0eac119f00 0x7f0eac11c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eb21ea700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac11c8c0 con 0x7f0eac117170 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eb21ea700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac11da60 con 0x7f0eac119f00 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eaaffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0eac119f00 0x7f0eac11c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eab7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0eac117170 0x7f0eac117550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57478/0 (socket says 172.21.15.191:57478) 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eab7fe700 1 -- 172.21.15.191:0/1032515276 learned_addr learned my addr 172.21.15.191:0/1032515276 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eaa7fc700 1 -- 172.21.15.191:0/1032515276 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208058144 0 0) 0x7f0eac11c8c0 con 0x7f0eac117170 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eaa7fc700 1 -- 172.21.15.191:0/1032515276 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e980035b0 con 0x7f0eac117170 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eaaffd700 1 -- 172.21.15.191:0/1032515276 >> v1:172.21.15.138:6789/0 conn(0x7f0eac117170 legacy=0x7f0eac117550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.550+0000 7f0eaaffd700 1 -- 172.21.15.191:0/1032515276 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac11dbd0 con 0x7f0eac119f00 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eaaffd700 1 --2- 172.21.15.191:0/1032515276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac119f00 0x7f0eac11c2f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0e940097c0 tx=0x7f0e94009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6022905f79e00350 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eaa7fc700 1 -- 172.21.15.191:0/1032515276 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e94004160 con 0x7f0eac119f00 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eaa7fc700 1 -- 172.21.15.191:0/1032515276 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e940042c0 con 0x7f0eac119f00 2024-04-27T04:33:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eaa7fc700 1 -- 172.21.15.191:0/1032515276 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e94004580 con 0x7f0eac119f00 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1032515276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac119f00 msgr2=0x7f0eac11c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 --2- 172.21.15.191:0/1032515276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac119f00 0x7f0eac11c2f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0e940097c0 tx=0x7f0e94009ad0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1032515276 shutdown_connections 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 --2- 172.21.15.191:0/1032515276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac119f00 0x7f0eac11c2f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1032515276 >> 172.21.15.191:0/1032515276 conn(0x7f0eac10cf60 msgr2=0x7f0eac10f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1032515276 shutdown_connections 2024-04-27T04:33:59.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.551+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1032515276 wait complete. 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.552+0000 7f0eb21ea700 1 Processor -- start 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.552+0000 7f0eb21ea700 1 -- start start 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.552+0000 7f0eb21ea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 0x7f0eac1ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.552+0000 7f0eb21ea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1ab830 con 0x7f0eac117170 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eab7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 0x7f0eac1ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eab7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 0x7f0eac1ab260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37238/0 (socket says 172.21.15.191:37238) 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eab7fe700 1 -- 172.21.15.191:0/1034260589 learned_addr learned my addr 172.21.15.191:0/1034260589 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eab7fe700 1 -- 172.21.15.191:0/1034260589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e940094a0 con 0x7f0eac117170 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eab7fe700 1 --2- 172.21.15.191:0/1034260589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 0x7f0eac1ab260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00cb30 tx=0x7f0e9c00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c00aa00 con 0x7f0eac117170 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00ab80 con 0x7f0eac117170 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac1aba30 con 0x7f0eac117170 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c00ae60 con 0x7f0eac117170 2024-04-27T04:33:59.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac1a52d0 con 0x7f0eac117170 2024-04-27T04:33:59.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0e9c01c440 con 0x7f0eac117170 2024-04-27T04:33:59.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.553+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e9c0129a0 con 0x7f0eac117170 2024-04-27T04:33:59.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.554+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac04f400 con 0x7f0eac117170 2024-04-27T04:33:59.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.556+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0e9c017070 con 0x7f0eac117170 2024-04-27T04:33:59.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.639+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "74a1ee1e-2f1c-4eaa-860a-ad1e52733bde", "id": 0} v 0) v1 -- 0x7f0eac11a6d0 con 0x7f0eac117170 2024-04-27T04:33:59.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.641+0000 7f0ea3fff700 1 -- 172.21.15.191:0/1034260589 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74a1ee1e-2f1c-4eaa-860a-ad1e52733bde", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0eac11a6d0 con 0x7f0eac117170 2024-04-27T04:33:59.642 INFO:teuthology.orchestra.run.smithi191.stdout:0 2024-04-27T04:33:59.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 msgr2=0x7f0eac1ab260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 --2- 172.21.15.191:0/1034260589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 0x7f0eac1ab260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00cb30 tx=0x7f0e9c00cef0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 shutdown_connections 2024-04-27T04:33:59.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 --2- 172.21.15.191:0/1034260589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac117170 0x7f0eac1ab260 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 >> 172.21.15.191:0/1034260589 conn(0x7f0eac10cf60 msgr2=0x7f0eac11bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:33:59.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 shutdown_connections 2024-04-27T04:33:59.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.642+0000 7f0eb21ea700 1 -- 172.21.15.191:0/1034260589 wait complete. 2024-04-27T04:33:59.655 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new d22f26e1-ca4f-4271-994f-b7011b3f030c 1 2024-04-27T04:33:59.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.777+0000 7f2f1d402700 1 Processor -- start 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.777+0000 7f2f1d402700 1 -- start start 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.777+0000 7f2f1d402700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f1806c140 0x7f2f1806c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.777+0000 7f2f1d402700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1806cf10 con 0x7f2f1806ca60 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.777+0000 7f2f1d402700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1811d930 con 0x7f2f1806c140 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f16ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f1806c140 0x7f2f1806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f16ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f1806c140 0x7f2f1806c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37248/0 (socket says 172.21.15.191:37248) 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f16ffd700 1 -- 172.21.15.191:0/1854809539 learned_addr learned my addr 172.21.15.191:0/1854809539 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f16ffd700 1 -- 172.21.15.191:0/1854809539 >> v1:172.21.15.138:6789/0 conn(0x7f2f1806ca60 legacy=0x7f2f1811b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f16ffd700 1 -- 172.21.15.191:0/1854809539 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1811da70 con 0x7f2f1806c140 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f15ffb700 1 -- 172.21.15.191:0/1854809539 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438613770 0 0) 0x7f2f1806cf10 con 0x7f2f1806ca60 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f16ffd700 1 --2- 172.21.15.191:0/1854809539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f1806c520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009870 tx=0x7f2f08009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175acc6e36dbfd0d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:33:59.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f15ffb700 1 -- 172.21.15.191:0/1854809539 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f08004160 con 0x7f2f1806c140 2024-04-27T04:33:59.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f15ffb700 1 -- 172.21.15.191:0/1854809539 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f080042c0 con 0x7f2f1806c140 2024-04-27T04:33:59.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.778+0000 7f2f15ffb700 1 -- 172.21.15.191:0/1854809539 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f08004580 con 0x7f2f1806c140 2024-04-27T04:33:59.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 -- 172.21.15.191:0/1854809539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 msgr2=0x7f2f1806c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 --2- 172.21.15.191:0/1854809539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f1806c520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009870 tx=0x7f2f08009b80 comp rx=0 tx=0).stop 2024-04-27T04:33:59.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 -- 172.21.15.191:0/1854809539 shutdown_connections 2024-04-27T04:33:59.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 --2- 172.21.15.191:0/1854809539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f1806c520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 -- 172.21.15.191:0/1854809539 >> 172.21.15.191:0/1854809539 conn(0x7f2f18070df0 msgr2=0x7f2f18071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 -- 172.21.15.191:0/1854809539 shutdown_connections 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.779+0000 7f2f1d402700 1 -- 172.21.15.191:0/1854809539 wait complete. 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f1d402700 1 Processor -- start 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f1d402700 1 -- start start 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f1d402700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f181ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f1d402700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181ab6d0 con 0x7f2f1806c140 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f16ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f181ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f16ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f181ab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37260/0 (socket says 172.21.15.191:37260) 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.780+0000 7f2f16ffd700 1 -- 172.21.15.191:0/2179389905 learned_addr learned my addr 172.21.15.191:0/2179389905 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2f16ffd700 1 -- 172.21.15.191:0/2179389905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08009550 con 0x7f2f1806c140 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2f16ffd700 1 --2- 172.21.15.191:0/2179389905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f181ab100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800a010 tx=0x7f2f08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f08003cf0 con 0x7f2f1806c140 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0801f070 con 0x7f2f1806c140 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f181ab870 con 0x7f2f1806c140 2024-04-27T04:33:59.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f08011630 con 0x7f2f1806c140 2024-04-27T04:33:59.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f181a5170 con 0x7f2f1806c140 2024-04-27T04:33:59.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2f08021030 con 0x7f2f1806c140 2024-04-27T04:33:59.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.781+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2f080248e0 con 0x7f2f1806c140 2024-04-27T04:33:59.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.782+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1804f0d0 con 0x7f2f1806c140 2024-04-27T04:33:59.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.784+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f2f08024480 con 0x7f2f1806c140 2024-04-27T04:33:59.821 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:33:59.819+0000 7f77976a0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T04:33:59.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.868+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d22f26e1-ca4f-4271-994f-b7011b3f030c", "id": 1} v 0) v1 -- 0x7f2f1806ca60 con 0x7f2f1806c140 2024-04-27T04:33:59.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.869+0000 7f2eff7fe700 1 -- 172.21.15.191:0/2179389905 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d22f26e1-ca4f-4271-994f-b7011b3f030c", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2f08024af0 con 0x7f2f1806c140 2024-04-27T04:33:59.871 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.870+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 msgr2=0x7f2f181ab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.870+0000 7f2f1d402700 1 --2- 172.21.15.191:0/2179389905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f181ab100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800a010 tx=0x7f2f08004750 comp rx=0 tx=0).stop 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.870+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 shutdown_connections 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.870+0000 7f2f1d402700 1 --2- 172.21.15.191:0/2179389905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1806c140 0x7f2f181ab100 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.870+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 >> 172.21.15.191:0/2179389905 conn(0x7f2f18070df0 msgr2=0x7f2f1811c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.871+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 shutdown_connections 2024-04-27T04:33:59.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:33:59.871+0000 7f2f1d402700 1 -- 172.21.15.191:0/2179389905 wait complete. 2024-04-27T04:33:59.883 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 699aaac0-41d4-47af-ae91-100b87b28b45 2 2024-04-27T04:34:00.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.004+0000 7fdda6e77700 1 Processor -- start 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdda6e77700 1 -- start start 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdda6e77700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdda0117d70 0x7fdda011a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdda6e77700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda011a730 con 0x7fdda01150e0 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdda6e77700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda011b8f0 con 0x7fdda0117d70 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdd9ffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdda0117d70 0x7fdda011a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdd9ffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdda0117d70 0x7fdda011a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37268/0 (socket says 172.21.15.191:37268) 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdd9ffff700 1 -- 172.21.15.191:0/3429701524 learned_addr learned my addr 172.21.15.191:0/3429701524 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:00.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdd9ffff700 1 -- 172.21.15.191:0/3429701524 >> v1:172.21.15.138:6789/0 conn(0x7fdda01150e0 legacy=0x7fdda01154c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdd9ffff700 1 -- 172.21.15.191:0/3429701524 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda011ba60 con 0x7fdda0117d70 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.005+0000 7fdd9ffff700 1 --2- 172.21.15.191:0/3429701524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda0117d70 0x7fdda011a160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009870 tx=0x7fdd8c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ed296c00c4067c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdd9f7fe700 1 -- 172.21.15.191:0/3429701524 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd8c004160 con 0x7fdda0117d70 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdd9f7fe700 1 -- 172.21.15.191:0/3429701524 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c0042c0 con 0x7fdda0117d70 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdd9f7fe700 1 -- 172.21.15.191:0/3429701524 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd8c004580 con 0x7fdda0117d70 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 -- 172.21.15.191:0/3429701524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda0117d70 msgr2=0x7fdda011a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 --2- 172.21.15.191:0/3429701524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda0117d70 0x7fdda011a160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009870 tx=0x7fdd8c009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 -- 172.21.15.191:0/3429701524 shutdown_connections 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 --2- 172.21.15.191:0/3429701524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda0117d70 0x7fdda011a160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 -- 172.21.15.191:0/3429701524 >> 172.21.15.191:0/3429701524 conn(0x7fdda0070e00 msgr2=0x7fdda0071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 -- 172.21.15.191:0/3429701524 shutdown_connections 2024-04-27T04:34:00.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.006+0000 7fdda6e77700 1 -- 172.21.15.191:0/3429701524 wait complete. 2024-04-27T04:34:00.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda6e77700 1 Processor -- start 2024-04-27T04:34:00.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda6e77700 1 -- start start 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda6e77700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 0x7fdda01a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda6e77700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01a9950 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda4c13700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 0x7fdda01a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda4c13700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 0x7fdda01a9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37274/0 (socket says 172.21.15.191:37274) 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.007+0000 7fdda4c13700 1 -- 172.21.15.191:0/1114035630 learned_addr learned my addr 172.21.15.191:0/1114035630 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdda4c13700 1 -- 172.21.15.191:0/1114035630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009550 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdda4c13700 1 --2- 172.21.15.191:0/1114035630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 0x7fdda01a9380 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c480 tx=0x7fdd9400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd94005200 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94005380 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd94006860 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda01a9af0 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01a33f0 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdd94003980 con 0x7fdda01150e0 2024-04-27T04:34:00.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.008+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fdd94011cc0 con 0x7fdda01150e0 2024-04-27T04:34:00.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.009+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda004f400 con 0x7fdda01150e0 2024-04-27T04:34:00.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.011+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdd94020060 con 0x7fdda01150e0 2024-04-27T04:34:00.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.095+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "699aaac0-41d4-47af-ae91-100b87b28b45", "id": 2} v 0) v1 -- 0x7fdda0117d70 con 0x7fdda01150e0 2024-04-27T04:34:00.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.097+0000 7fdd9d7fa700 1 -- 172.21.15.191:0/1114035630 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "699aaac0-41d4-47af-ae91-100b87b28b45", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdd94003dc0 con 0x7fdda01150e0 2024-04-27T04:34:00.098 INFO:teuthology.orchestra.run.smithi191.stdout:2 2024-04-27T04:34:00.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 msgr2=0x7fdda01a9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 --2- 172.21.15.191:0/1114035630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 0x7fdda01a9380 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c480 tx=0x7fdd9400c840 comp rx=0 tx=0).stop 2024-04-27T04:34:00.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 shutdown_connections 2024-04-27T04:34:00.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 --2- 172.21.15.191:0/1114035630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda01150e0 0x7fdda01a9380 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 >> 172.21.15.191:0/1114035630 conn(0x7fdda0070e00 msgr2=0x7fdda0118150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:00.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 shutdown_connections 2024-04-27T04:34:00.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.098+0000 7fdda6e77700 1 -- 172.21.15.191:0/1114035630 wait complete. 2024-04-27T04:34:00.111 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new cecc3ff7-644a-480d-ba0b-ac744a496062 3 2024-04-27T04:34:00.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.230+0000 7fd7a3b80700 1 Processor -- start 2024-04-27T04:34:00.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.230+0000 7fd7a3b80700 1 -- start start 2024-04-27T04:34:00.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.230+0000 7fd7a3b80700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd79c117d70 0x7fd79c11a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:00.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.230+0000 7fd7a3b80700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c11a730 con 0x7fd79c1150e0 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.230+0000 7fd7a3b80700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c11b8f0 con 0x7fd79c117d70 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a111b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd79c117d70 0x7fd79c11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a111b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd79c117d70 0x7fd79c11a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37284/0 (socket says 172.21.15.191:37284) 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a111b700 1 -- 172.21.15.191:0/1607824573 learned_addr learned my addr 172.21.15.191:0/1607824573 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a191c700 1 --1- 172.21.15.191:0/1607824573 >> v1:172.21.15.138:6789/0 conn(0x7fd79c1150e0 0x7fd79c1154c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57502/0 (socket says 172.21.15.191:57502) 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a111b700 1 -- 172.21.15.191:0/1607824573 >> v1:172.21.15.138:6789/0 conn(0x7fd79c1150e0 legacy=0x7fd79c1154c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a111b700 1 -- 172.21.15.191:0/1607824573 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c11ba60 con 0x7fd79c117d70 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a091a700 1 -- 172.21.15.191:0/1607824573 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769087762 0 0) 0x7fd79c11a730 con 0x7fd79c1150e0 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a111b700 1 --2- 172.21.15.191:0/1607824573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c117d70 0x7fd79c11a160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009870 tx=0x7fd78c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a6c60997970a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a091a700 1 -- 172.21.15.191:0/1607824573 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd78c004160 con 0x7fd79c117d70 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a091a700 1 -- 172.21.15.191:0/1607824573 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c0042c0 con 0x7fd79c117d70 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.231+0000 7fd7a091a700 1 -- 172.21.15.191:0/1607824573 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd78c004580 con 0x7fd79c117d70 2024-04-27T04:34:00.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 -- 172.21.15.191:0/1607824573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c117d70 msgr2=0x7fd79c11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 --2- 172.21.15.191:0/1607824573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c117d70 0x7fd79c11a160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009870 tx=0x7fd78c009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:00.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 -- 172.21.15.191:0/1607824573 shutdown_connections 2024-04-27T04:34:00.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 --2- 172.21.15.191:0/1607824573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c117d70 0x7fd79c11a160 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 -- 172.21.15.191:0/1607824573 >> 172.21.15.191:0/1607824573 conn(0x7fd79c070e00 msgr2=0x7fd79c071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:00.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 -- 172.21.15.191:0/1607824573 shutdown_connections 2024-04-27T04:34:00.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.232+0000 7fd7a3b80700 1 -- 172.21.15.191:0/1607824573 wait complete. 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a3b80700 1 Processor -- start 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a3b80700 1 -- start start 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a3b80700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 0x7fd79c1a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a3b80700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c1a9780 con 0x7fd79c1150e0 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a191c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 0x7fd79c1a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a191c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 0x7fd79c1a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37290/0 (socket says 172.21.15.191:37290) 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a191c700 1 -- 172.21.15.191:0/436150029 learned_addr learned my addr 172.21.15.191:0/436150029 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.233+0000 7fd7a191c700 1 -- 172.21.15.191:0/436150029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c009550 con 0x7fd79c1150e0 2024-04-27T04:34:00.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7a191c700 1 --2- 172.21.15.191:0/436150029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 0x7fd79c1a91b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c4c0 tx=0x7fd79800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd798004bb0 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798004d10 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c1a9980 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7980070d0 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c1a32a0 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd79801d3f0 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.234+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd798013850 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.235+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c04f400 con 0x7fd79c1150e0 2024-04-27T04:34:00.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.237+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd7980133c0 con 0x7fd79c1150e0 2024-04-27T04:34:00.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.319+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cecc3ff7-644a-480d-ba0b-ac744a496062", "id": 3} v 0) v1 -- 0x7fd79c117d70 con 0x7fd79c1150e0 2024-04-27T04:34:00.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.323+0000 7fd7927fc700 1 -- 172.21.15.191:0/436150029 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cecc3ff7-644a-480d-ba0b-ac744a496062", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd7980136b0 con 0x7fd79c1150e0 2024-04-27T04:34:00.324 INFO:teuthology.orchestra.run.smithi191.stdout:3 2024-04-27T04:34:00.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 msgr2=0x7fd79c1a91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 --2- 172.21.15.191:0/436150029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 0x7fd79c1a91b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c4c0 tx=0x7fd79800c880 comp rx=0 tx=0).stop 2024-04-27T04:34:00.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 shutdown_connections 2024-04-27T04:34:00.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 --2- 172.21.15.191:0/436150029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c1150e0 0x7fd79c1a91b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 >> 172.21.15.191:0/436150029 conn(0x7fd79c070e00 msgr2=0x7fd79c119ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:00.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 shutdown_connections 2024-04-27T04:34:00.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:00.324+0000 7fd7a3b80700 1 -- 172.21.15.191:0/436150029 wait complete. 2024-04-27T04:34:00.334 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:00.332+0000 7f77976a0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-27T04:34:00.337 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-27T04:34:00.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-27T04:34:00.339 INFO:tasks.ceph.osd.0:Started 2024-04-27T04:34:00.339 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-27T04:34:00.340 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-27T04:34:00.342 INFO:tasks.ceph.osd.1:Started 2024-04-27T04:34:00.342 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-27T04:34:00.342 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-27T04:34:00.344 INFO:tasks.ceph.osd.2:Started 2024-04-27T04:34:00.344 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-27T04:34:00.344 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-27T04:34:00.347 INFO:tasks.ceph.osd.3:Started 2024-04-27T04:34:00.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:34:00.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d8527700 1 Processor -- start 2024-04-27T04:34:00.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d8527700 1 -- start start 2024-04-27T04:34:00.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d8527700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6d00946c0 0x7fb6d0094aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d8527700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d0067490 con 0x7fb6d0094fe0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d8527700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d0095740 con 0x7fb6d00946c0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d62c3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6d00946c0 0x7fb6d0094aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d5ac2700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6d0094fe0 0x7fb6d012ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59786/0 (socket says 172.21.15.138:59786) 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d5ac2700 1 -- 172.21.15.138:0/2971062826 learned_addr learned my addr 172.21.15.138:0/2971062826 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d62c3700 1 -- 172.21.15.138:0/2971062826 >> v1:172.21.15.138:6789/0 conn(0x7fb6d0094fe0 legacy=0x7fb6d012ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d62c3700 1 -- 172.21.15.138:0/2971062826 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d009db40 con 0x7fb6d00946c0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.474+0000 7fb6d52c1700 1 -- 172.21.15.138:0/2971062826 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179625942 0 0) 0x7fb6d0067490 con 0x7fb6d0094fe0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d62c3700 1 --2- 172.21.15.138:0/2971062826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d0094aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00097c0 tx=0x7fb6c0009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699adf02c71c5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d52c1700 1 -- 172.21.15.138:0/2971062826 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0004160 con 0x7fb6d00946c0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d52c1700 1 -- 172.21.15.138:0/2971062826 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c00042c0 con 0x7fb6d00946c0 2024-04-27T04:34:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d52c1700 1 -- 172.21.15.138:0/2971062826 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0004580 con 0x7fb6d00946c0 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 -- 172.21.15.138:0/2971062826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 msgr2=0x7fb6d0094aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 --2- 172.21.15.138:0/2971062826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d0094aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00097c0 tx=0x7fb6c0009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 -- 172.21.15.138:0/2971062826 shutdown_connections 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 --2- 172.21.15.138:0/2971062826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d0094aa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 -- 172.21.15.138:0/2971062826 >> 172.21.15.138:0/2971062826 conn(0x7fb6d009d020 msgr2=0x7fb6d009d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 -- 172.21.15.138:0/2971062826 shutdown_connections 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.475+0000 7fb6d8527700 1 -- 172.21.15.138:0/2971062826 wait complete. 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.476+0000 7fb6d8527700 1 Processor -- start 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.476+0000 7fb6d8527700 1 -- start start 2024-04-27T04:34:00.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.476+0000 7fb6d8527700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d01a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.476+0000 7fb6d8527700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d01a8450 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d62c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d01a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d62c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d01a6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59072/0 (socket says 172.21.15.138:59072) 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d62c3700 1 -- 172.21.15.138:0/789810153 learned_addr learned my addr 172.21.15.138:0/789810153 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d62c3700 1 -- 172.21.15.138:0/789810153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00094a0 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d62c3700 1 --2- 172.21.15.138:0/789810153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d01a6bb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006ec0 tx=0x7fb6c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0011630 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c001e070 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d01a81b0 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c00117b0 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d01a9c00 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb6c0011c60 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.477+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb6c001a940 con 0x7fb6d00946c0 2024-04-27T04:34:00.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.478+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d01a9ef0 con 0x7fb6d00946c0 2024-04-27T04:34:00.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.480+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb6c0020070 con 0x7fb6d00946c0 2024-04-27T04:34:00.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.547+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d0062510 con 0x7fb6d00946c0 2024-04-27T04:34:00.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6c6ffd700 1 -- 172.21.15.138:0/789810153 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7fb6c001ac50 con 0x7fb6d00946c0 2024-04-27T04:34:00.549 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:00.550 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":6,"fsid":"512e09fb-564f-4fb6-aa0b-f541dbeb017d","created":"2024-04-27T04:33:59.121696+0000","modified":"2024-04-27T04:34:00.320575+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T04:34:00.320575+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"74a1ee1e-2f1c-4eaa-860a-ad1e52733bde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d22f26e1-ca4f-4271-994f-b7011b3f030c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"699aaac0-41d4-47af-ae91-100b87b28b45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cecc3ff7-644a-480d-ba0b-ac744a496062","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:34:00.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 msgr2=0x7fb6d01a6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:00.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 --2- 172.21.15.138:0/789810153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d01a6bb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006ec0 tx=0x7fb6c0004750 comp rx=0 tx=0).stop 2024-04-27T04:34:00.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 shutdown_connections 2024-04-27T04:34:00.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 --2- 172.21.15.138:0/789810153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d00946c0 0x7fb6d01a6bb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:00.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 >> 172.21.15.138:0/789810153 conn(0x7fb6d009d020 msgr2=0x7fb6d0071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:00.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 shutdown_connections 2024-04-27T04:34:00.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:00.548+0000 7fb6d8527700 1 -- 172.21.15.138:0/789810153 wait complete. 2024-04-27T04:34:00.561 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-27T04:34:00.562 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-27T04:34:00.564 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:00.563+0000 7f77976a0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-27T04:34:00.936 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-04-27T04:34:00.935+0000 7f44c6a6a7c0 -1 Falling back to public interface 2024-04-27T04:34:00.946 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-04-27T04:34:00.945+0000 7f1259c0d7c0 -1 Falling back to public interface 2024-04-27T04:34:00.950 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-04-27T04:34:00.949+0000 7f6d703597c0 -1 Falling back to public interface 2024-04-27T04:34:00.951 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:00.950+0000 7f77976a0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T04:34:00.959 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-04-27T04:34:00.958+0000 7fd71ade87c0 -1 Falling back to public interface 2024-04-27T04:34:00.965 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.074+0000 7f265d41a700 1 Processor -- start 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.074+0000 7f265d41a700 1 -- start start 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f265d41a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f265812a9e0 0x7f265812adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f265d41a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265806ed40 con 0x7f265812d830 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f265d41a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265806f980 con 0x7f265812a9e0 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2656ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f265812a9e0 0x7f265812adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2656ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f265812a9e0 0x7f265812adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59084/0 (socket says 172.21.15.138:59084) 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f26567fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f265812d830 0x7f265812dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59792/0 (socket says 172.21.15.138:59792) 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2656ffd700 1 -- 172.21.15.138:0/441120946 learned_addr learned my addr 172.21.15.138:0/441120946 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2656ffd700 1 -- 172.21.15.138:0/441120946 >> v1:172.21.15.138:6789/0 conn(0x7f265812d830 legacy=0x7f265812dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:01.078 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2656ffd700 1 -- 172.21.15.138:0/441120946 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265806f2c0 con 0x7f265812a9e0 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2655ffb700 1 -- 172.21.15.138:0/441120946 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799796967 0 0) 0x7f265806ed40 con 0x7f265812d830 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2656ffd700 1 --2- 172.21.15.138:0/441120946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265812adc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f264c009870 tx=0x7f264c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a10ba7b7856e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2655ffb700 1 -- 172.21.15.138:0/441120946 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f264c004160 con 0x7f265812a9e0 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2655ffb700 1 -- 172.21.15.138:0/441120946 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c0042c0 con 0x7f265812a9e0 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.075+0000 7f2655ffb700 1 -- 172.21.15.138:0/441120946 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f264c004580 con 0x7f265812a9e0 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 -- 172.21.15.138:0/441120946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 msgr2=0x7f265812adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 --2- 172.21.15.138:0/441120946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265812adc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f264c009870 tx=0x7f264c009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 -- 172.21.15.138:0/441120946 shutdown_connections 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 --2- 172.21.15.138:0/441120946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265812adc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 -- 172.21.15.138:0/441120946 >> 172.21.15.138:0/441120946 conn(0x7f265809c040 msgr2=0x7f265809db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 -- 172.21.15.138:0/441120946 shutdown_connections 2024-04-27T04:34:01.079 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.076+0000 7f265d41a700 1 -- 172.21.15.138:0/441120946 wait complete. 2024-04-27T04:34:01.081 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f265d41a700 1 Processor -- start 2024-04-27T04:34:01.081 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f265d41a700 1 -- start start 2024-04-27T04:34:01.081 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f265d41a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265811a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:01.081 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f265d41a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265811c6a0 con 0x7f265812a9e0 2024-04-27T04:34:01.081 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f2656ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265811a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f2656ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265811a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59098/0 (socket says 172.21.15.138:59098) 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.077+0000 7f2656ffd700 1 -- 172.21.15.138:0/2138197771 learned_addr learned my addr 172.21.15.138:0/2138197771 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f2656ffd700 1 -- 172.21.15.138:0/2138197771 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c009550 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f2656ffd700 1 --2- 172.21.15.138:0/2138197771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265811a610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f264c00a010 tx=0x7f264c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f264c003cf0 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c01f070 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265811d010 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f264c011630 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265811be70 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f264c021030 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.078+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f264c024de0 con 0x7f265812a9e0 2024-04-27T04:34:01.082 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.079+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2658067b70 con 0x7f265812a9e0 2024-04-27T04:34:01.083 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.081+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f264c011bd0 con 0x7f265812a9e0 2024-04-27T04:34:01.148 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.147+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2658062510 con 0x7f265812a9e0 2024-04-27T04:34:01.149 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.147+0000 7f263f7fe700 1 -- 172.21.15.138:0/2138197771 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7f264c011d70 con 0x7f265812a9e0 2024-04-27T04:34:01.149 INFO:teuthology.misc.health.smithi138.stdout: 2024-04-27T04:34:01.149 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":6,"fsid":"512e09fb-564f-4fb6-aa0b-f541dbeb017d","created":"2024-04-27T04:33:59.121696+0000","modified":"2024-04-27T04:34:00.320575+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T04:34:00.320575+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"74a1ee1e-2f1c-4eaa-860a-ad1e52733bde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d22f26e1-ca4f-4271-994f-b7011b3f030c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"699aaac0-41d4-47af-ae91-100b87b28b45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cecc3ff7-644a-480d-ba0b-ac744a496062","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:34:01.150 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 msgr2=0x7f265811a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:01.150 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 --2- 172.21.15.138:0/2138197771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265811a610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f264c00a010 tx=0x7f264c004750 comp rx=0 tx=0).stop 2024-04-27T04:34:01.150 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 shutdown_connections 2024-04-27T04:34:01.150 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 --2- 172.21.15.138:0/2138197771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f265812a9e0 0x7f265811a610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:01.150 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 >> 172.21.15.138:0/2138197771 conn(0x7f265809c040 msgr2=0x7f2658070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:01.151 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 shutdown_connections 2024-04-27T04:34:01.151 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:01.148+0000 7f265d41a700 1 -- 172.21.15.138:0/2138197771 wait complete. 2024-04-27T04:34:01.160 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-27T04:34:01.629 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:01.628+0000 7f77976a0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T04:34:01.702 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:01.700+0000 7f77976a0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T04:34:01.835 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:01.834+0000 7f77976a0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T04:34:02.010 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.009+0000 7f77976a0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T04:34:02.086 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.085+0000 7f77976a0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T04:34:02.164 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.162+0000 7f77976a0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T04:34:02.240 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.239+0000 7f77976a0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T04:34:02.335 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.334+0000 7f77976a0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T04:34:02.531 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.530+0000 7f77976a0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T04:34:02.747 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.746+0000 7f77976a0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T04:34:02.904 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.903+0000 7f77976a0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T04:34:02.975 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:02.974+0000 7f77976a0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T04:34:03.670 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:03.668+0000 7f77976a0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T04:34:03.874 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:03.873+0000 7f77976a0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T04:34:04.034 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:04.033+0000 7f77976a0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T04:34:04.372 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-04-27T04:34:04.372+0000 7f44c6a6a7c0 -1 osd.0 0 log_to_monitors true 2024-04-27T04:34:04.438 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:04.436+0000 7f77976a0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T04:34:04.470 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-04-27T04:34:04.469+0000 7f1259c0d7c0 -1 osd.3 0 log_to_monitors true 2024-04-27T04:34:04.477 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-04-27T04:34:04.476+0000 7fd71ade87c0 -1 osd.2 0 log_to_monitors true 2024-04-27T04:34:04.480 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-04-27T04:34:04.479+0000 7f6d703597c0 -1 osd.1 0 log_to_monitors true 2024-04-27T04:34:04.615 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:04.614+0000 7f77976a0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T04:34:04.974 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:04.973+0000 7f77976a0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T04:34:05.073 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:05.072+0000 7f77976a0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T04:34:06.094 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-04-27T04:34:06.093+0000 7f7758a90700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-27T04:34:06.254 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-04-27T04:34:06.254+0000 7f44ba60b700 -1 osd.0 0 waiting for initial osdmap 2024-04-27T04:34:06.264 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-04-27T04:34:06.264+0000 7f44b29e6700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:34:07.564 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.685+0000 7fcf5a158700 1 Processor -- start 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.685+0000 7fcf5a158700 1 -- start start 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf5a158700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf54130b40 0x7fcf54132f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf5a158700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5406ed40 con 0x7fcf5412a9e0 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf5a158700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5406f980 con 0x7fcf54130b40 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf52ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf54130b40 0x7fcf54132f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf537fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcf5412a9e0 0x7fcf5412adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59798/0 (socket says 172.21.15.138:59798) 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf52ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf54130b40 0x7fcf54132f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59124/0 (socket says 172.21.15.138:59124) 2024-04-27T04:34:07.689 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf537fe700 1 -- 172.21.15.138:0/2346630823 learned_addr learned my addr 172.21.15.138:0/2346630823 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf52ffd700 1 -- 172.21.15.138:0/2346630823 >> v1:172.21.15.138:6789/0 conn(0x7fcf5412a9e0 legacy=0x7fcf5412adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf52ffd700 1 -- 172.21.15.138:0/2346630823 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5406f2c0 con 0x7fcf54130b40 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf527fc700 1 -- 172.21.15.138:0/2346630823 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592791935 0 0) 0x7fcf5406ed40 con 0x7fcf5412a9e0 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.686+0000 7fcf52ffd700 1 --2- 172.21.15.138:0/2346630823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54130b40 0x7fcf54132f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0097c0 tx=0x7fcf3c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2de9fbac019f3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf527fc700 1 -- 172.21.15.138:0/2346630823 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf3c004160 con 0x7fcf54130b40 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf527fc700 1 -- 172.21.15.138:0/2346630823 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c0042c0 con 0x7fcf54130b40 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf527fc700 1 -- 172.21.15.138:0/2346630823 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf3c004580 con 0x7fcf54130b40 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 -- 172.21.15.138:0/2346630823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54130b40 msgr2=0x7fcf54132f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 --2- 172.21.15.138:0/2346630823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54130b40 0x7fcf54132f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0097c0 tx=0x7fcf3c009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 -- 172.21.15.138:0/2346630823 shutdown_connections 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 --2- 172.21.15.138:0/2346630823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54130b40 0x7fcf54132f30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:07.690 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 -- 172.21.15.138:0/2346630823 >> 172.21.15.138:0/2346630823 conn(0x7fcf5409c040 msgr2=0x7fcf5409db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:07.691 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 -- 172.21.15.138:0/2346630823 shutdown_connections 2024-04-27T04:34:07.691 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.687+0000 7fcf5a158700 1 -- 172.21.15.138:0/2346630823 wait complete. 2024-04-27T04:34:07.691 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.688+0000 7fcf5a158700 1 Processor -- start 2024-04-27T04:34:07.691 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.688+0000 7fcf5a158700 1 -- start start 2024-04-27T04:34:07.693 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf5a158700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 0x7fcf54118e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:07.693 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf5a158700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5409a040 con 0x7fcf5412a9e0 2024-04-27T04:34:07.693 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 0x7fcf54118e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:07.693 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 0x7fcf54118e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59130/0 (socket says 172.21.15.138:59130) 2024-04-27T04:34:07.693 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf537fe700 1 -- 172.21.15.138:0/3236765691 learned_addr learned my addr 172.21.15.138:0/3236765691 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:07.693 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf537fe700 1 -- 172.21.15.138:0/3236765691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0094a0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf537fe700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 0x7fcf54118e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400c4a0 tx=0x7fcf4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf44004bb0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44004d10 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5411a6a0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf440070d0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.689+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5411b2e0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.690+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 3) v1 ==== 44655+0+0 (secure 0 0 0) 0x7fcf440072e0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.690+0000 7fcf4bfff700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fcf34037630 0x7fcf34039af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.690+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54067b70 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.690+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1903+0+0 (secure 0 0 0) 0x7fcf440493e0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.690+0000 7fcf52ffd700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fcf34037630 0x7fcf34039af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:07.694 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.691+0000 7fcf52ffd700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fcf34037630 0x7fcf34039af0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf5404ed60 tx=0x7fcf3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:07.695 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.695+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fcf440102c0 con 0x7fcf5412a9e0 2024-04-27T04:34:07.887 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.886+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf54062510 con 0x7fcf5412a9e0 2024-04-27T04:34:07.888 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.887+0000 7fcf4bfff700 1 -- 172.21.15.138:0/3236765691 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+4600 (secure 0 0 0) 0x7fcf44011360 con 0x7fcf5412a9e0 2024-04-27T04:34:07.888 INFO:teuthology.misc.health.smithi138.stdout: 2024-04-27T04:34:07.888 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":9,"fsid":"512e09fb-564f-4fb6-aa0b-f541dbeb017d","created":"2024-04-27T04:33:59.121696+0000","modified":"2024-04-27T04:34:06.427716+0000","last_up_change":"2024-04-27T04:34:06.427716+0000","last_in_change":"2024-04-27T04:34:00.320575+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"74a1ee1e-2f1c-4eaa-860a-ad1e52733bde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":39496},{"type":"v1","addr":"172.21.15.138:6801","nonce":39496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":39496},{"type":"v1","addr":"172.21.15.138:6803","nonce":39496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":39496},{"type":"v1","addr":"172.21.15.138:6807","nonce":39496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":39496},{"type":"v1","addr":"172.21.15.138:6805","nonce":39496}]},"public_addr":"172.21.15.138:6801/39496","cluster_addr":"172.21.15.138:6803/39496","heartbeat_back_addr":"172.21.15.138:6807/39496","heartbeat_front_addr":"172.21.15.138:6805/39496","state":["exists","up"]},{"osd":1,"uuid":"d22f26e1-ca4f-4271-994f-b7011b3f030c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"699aaac0-41d4-47af-ae91-100b87b28b45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cecc3ff7-644a-480d-ba0b-ac744a496062","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:34:07.890 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fcf34037630 msgr2=0x7fcf34039af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:07.890 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fcf34037630 0x7fcf34039af0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf5404ed60 tx=0x7fcf3c008030 comp rx=0 tx=0).stop 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 msgr2=0x7fcf54118e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 0x7fcf54118e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400c4a0 tx=0x7fcf4400c860 comp rx=0 tx=0).stop 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 shutdown_connections 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fcf34037630 0x7fcf34039af0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 --2- 172.21.15.138:0/3236765691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf5412a9e0 0x7fcf54118e30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 >> 172.21.15.138:0/3236765691 conn(0x7fcf5409c040 msgr2=0x7fcf540998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 shutdown_connections 2024-04-27T04:34:07.891 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:07.889+0000 7fcf5a158700 1 -- 172.21.15.138:0/3236765691 wait complete. 2024-04-27T04:34:07.901 DEBUG:teuthology.misc:1 of 4 OSDs are up 2024-04-27T04:34:08.037 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-04-27T04:34:08.035+0000 7f124d7ae700 -1 osd.3 0 waiting for initial osdmap 2024-04-27T04:34:08.039 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-04-27T04:34:08.038+0000 7fd70e989700 -1 osd.2 0 waiting for initial osdmap 2024-04-27T04:34:08.049 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-04-27T04:34:08.048+0000 7f1245b89700 -1 osd.3 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:34:08.053 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-04-27T04:34:08.052+0000 7fd706d64700 -1 osd.2 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:34:08.079 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-04-27T04:34:08.078+0000 7f6d63efa700 -1 osd.1 0 waiting for initial osdmap 2024-04-27T04:34:08.093 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-04-27T04:34:08.092+0000 7f6d5c2d5700 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T04:34:14.304 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T04:34:14.428 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.425+0000 7f2cc1843700 1 Processor -- start 2024-04-27T04:34:14.428 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.425+0000 7f2cc1843700 1 -- start start 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cc1843700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2cbc12a020 0x7f2cbc12a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cc1843700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc06c810 con 0x7f2cbc12a940 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cc1843700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc06d560 con 0x7f2cbc12a020 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cbaffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2cbc12a020 0x7f2cbc12a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cba7fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2cbc12a940 0x7f2cbc093e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43692/0 (socket says 172.21.15.138:43692) 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cba7fc700 1 -- 172.21.15.138:0/4288284811 learned_addr learned my addr 172.21.15.138:0/4288284811 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cbaffd700 1 -- 172.21.15.138:0/4288284811 >> v1:172.21.15.138:6789/0 conn(0x7f2cbc12a940 legacy=0x7f2cbc093e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cbaffd700 1 -- 172.21.15.138:0/4288284811 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc095770 con 0x7f2cbc12a020 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.426+0000 7f2cbaffd700 1 --2- 172.21.15.138:0/4288284811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc12a400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097c0 tx=0x7f2cb0009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc6d97588f0628 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cb9ffb700 1 -- 172.21.15.138:0/4288284811 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cb0004160 con 0x7f2cbc12a020 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cb9ffb700 1 -- 172.21.15.138:0/4288284811 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb00042c0 con 0x7f2cbc12a020 2024-04-27T04:34:14.429 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cb9ffb700 1 -- 172.21.15.138:0/4288284811 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cb0004580 con 0x7f2cbc12a020 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 -- 172.21.15.138:0/4288284811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 msgr2=0x7f2cbc12a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 --2- 172.21.15.138:0/4288284811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc12a400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097c0 tx=0x7f2cb0009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 -- 172.21.15.138:0/4288284811 shutdown_connections 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 --2- 172.21.15.138:0/4288284811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc12a400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 -- 172.21.15.138:0/4288284811 >> 172.21.15.138:0/4288284811 conn(0x7f2cbc09c080 msgr2=0x7f2cbc09c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 -- 172.21.15.138:0/4288284811 shutdown_connections 2024-04-27T04:34:14.432 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.427+0000 7f2cc1843700 1 -- 172.21.15.138:0/4288284811 wait complete. 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.428+0000 7f2cc1843700 1 Processor -- start 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.428+0000 7f2cc1843700 1 -- start start 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cc1843700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc121560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cc1843700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc123f50 con 0x7f2cbc12a020 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cbaffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc121560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cbaffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc121560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47698/0 (socket says 172.21.15.138:47698) 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cbaffd700 1 -- 172.21.15.138:0/2078900811 learned_addr learned my addr 172.21.15.138:0/2078900811 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cbaffd700 1 -- 172.21.15.138:0/2078900811 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00094a0 con 0x7f2cbc12a020 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cbaffd700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc121560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009b10 tx=0x7f2cb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cb001d070 con 0x7f2cbc12a020 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0010600 con 0x7f2cbc12a020 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cb00108a0 con 0x7f2cbc12a020 2024-04-27T04:34:14.433 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc124a00 con 0x7f2cbc12a020 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.429+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc123470 con 0x7f2cbc12a020 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.430+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2cb0021070 con 0x7f2cbc12a020 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.430+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc123d80 con 0x7f2cbc12a020 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.430+0000 7f2ca3fff700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2ca8037790 0x7f2ca8039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.430+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f2cb004b160 con 0x7f2cbc12a020 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.431+0000 7f2cba7fc700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2ca8037790 0x7f2ca8039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:14.434 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.431+0000 7f2cba7fc700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2ca8037790 0x7f2ca8039c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc12ad90 tx=0x7f2cac007b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:14.437 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.435+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2cb000f960 con 0x7f2cbc12a020 2024-04-27T04:34:14.626 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.624+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cbc062510 con 0x7f2cbc12a020 2024-04-27T04:34:14.627 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.625+0000 7f2ca3fff700 1 -- 172.21.15.138:0/2078900811 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7728 (secure 0 0 0) 0x7f2cb0028070 con 0x7f2cbc12a020 2024-04-27T04:34:14.627 INFO:teuthology.misc.health.smithi138.stdout: 2024-04-27T04:34:14.627 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":13,"fsid":"512e09fb-564f-4fb6-aa0b-f541dbeb017d","created":"2024-04-27T04:33:59.121696+0000","modified":"2024-04-27T04:34:11.440569+0000","last_up_change":"2024-04-27T04:34:08.432381+0000","last_in_change":"2024-04-27T04:34:00.320575+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:34:09.130881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74a1ee1e-2f1c-4eaa-860a-ad1e52733bde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":39496},{"type":"v1","addr":"172.21.15.138:6801","nonce":39496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":39496},{"type":"v1","addr":"172.21.15.138:6803","nonce":39496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":39496},{"type":"v1","addr":"172.21.15.138:6807","nonce":39496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":39496},{"type":"v1","addr":"172.21.15.138:6805","nonce":39496}]},"public_addr":"172.21.15.138:6801/39496","cluster_addr":"172.21.15.138:6803/39496","heartbeat_back_addr":"172.21.15.138:6807/39496","heartbeat_front_addr":"172.21.15.138:6805/39496","state":["exists","up"]},{"osd":1,"uuid":"d22f26e1-ca4f-4271-994f-b7011b3f030c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":41886},{"type":"v1","addr":"172.21.15.191:6809","nonce":41886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":41886},{"type":"v1","addr":"172.21.15.191:6811","nonce":41886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":41886},{"type":"v1","addr":"172.21.15.191:6815","nonce":41886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":41886},{"type":"v1","addr":"172.21.15.191:6813","nonce":41886}]},"public_addr":"172.21.15.191:6809/41886","cluster_addr":"172.21.15.191:6811/41886","heartbeat_back_addr":"172.21.15.191:6815/41886","heartbeat_front_addr":"172.21.15.191:6813/41886","state":["exists","up"]},{"osd":2,"uuid":"699aaac0-41d4-47af-ae91-100b87b28b45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":41888},{"type":"v1","addr":"172.21.15.191:6817","nonce":41888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":41888},{"type":"v1","addr":"172.21.15.191:6819","nonce":41888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":41888},{"type":"v1","addr":"172.21.15.191:6823","nonce":41888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":41888},{"type":"v1","addr":"172.21.15.191:6821","nonce":41888}]},"public_addr":"172.21.15.191:6817/41888","cluster_addr":"172.21.15.191:6819/41888","heartbeat_back_addr":"172.21.15.191:6823/41888","heartbeat_front_addr":"172.21.15.191:6821/41888","state":["exists","up"]},{"osd":3,"uuid":"cecc3ff7-644a-480d-ba0b-ac744a496062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":41887},{"type":"v1","addr":"172.21.15.191:6801","nonce":41887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":41887},{"type":"v1","addr":"172.21.15.191:6803","nonce":41887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":41887},{"type":"v1","addr":"172.21.15.191:6807","nonce":41887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":41887},{"type":"v1","addr":"172.21.15.191:6805","nonce":41887}]},"public_addr":"172.21.15.191:6801/41887","cluster_addr":"172.21.15.191:6803/41887","heartbeat_back_addr":"172.21.15.191:6807/41887","heartbeat_front_addr":"172.21.15.191:6805/41887","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.420983+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.449423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.499228+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.424812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:34:14.629 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2ca8037790 msgr2=0x7f2ca8039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:14.629 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2ca8037790 0x7f2ca8039c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc12ad90 tx=0x7f2cac007b10 comp rx=0 tx=0).stop 2024-04-27T04:34:14.629 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 msgr2=0x7f2cbc121560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:14.629 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc121560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009b10 tx=0x7f2cb0004750 comp rx=0 tx=0).stop 2024-04-27T04:34:14.630 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 shutdown_connections 2024-04-27T04:34:14.630 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2ca8037790 0x7f2ca8039c50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:14.630 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 --2- 172.21.15.138:0/2078900811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc12a020 0x7f2cbc121560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:14.630 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 >> 172.21.15.138:0/2078900811 conn(0x7f2cbc09c080 msgr2=0x7f2cbc0977b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:14.630 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 shutdown_connections 2024-04-27T04:34:14.630 INFO:teuthology.misc.health.smithi138.stderr:2024-04-27T04:34:14.627+0000 7f2cc1843700 1 -- 172.21.15.138:0/2078900811 wait complete. 2024-04-27T04:34:14.640 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-27T04:34:14.640 INFO:tasks.ceph:Creating RBD pool 2024-04-27T04:34:14.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-27T04:34:14.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.757+0000 7f003535e700 1 Processor -- start 2024-04-27T04:34:14.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.757+0000 7f003535e700 1 -- start start 2024-04-27T04:34:14.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f003535e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00300939f0 0x7f0030093dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:14.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f003535e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003006d980 con 0x7f0030094310 2024-04-27T04:34:14.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f003535e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0030095a50 con 0x7f00300939f0 2024-04-27T04:34:14.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002effd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00300939f0 0x7f0030093dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002e7fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0030094310 0x7f0030129e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43704/0 (socket says 172.21.15.138:43704) 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002e7fc700 1 -- 172.21.15.138:0/3475834113 learned_addr learned my addr 172.21.15.138:0/3475834113 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002effd700 1 -- 172.21.15.138:0/3475834113 >> v1:172.21.15.138:6789/0 conn(0x7f0030094310 legacy=0x7f0030129e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002effd700 1 -- 172.21.15.138:0/3475834113 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003006d590 con 0x7f00300939f0 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002dffb700 1 -- 172.21.15.138:0/3475834113 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123970381 0 0) 0x7f003006d980 con 0x7f0030094310 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002effd700 1 --2- 172.21.15.138:0/3475834113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f0030093dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00240097c0 tx=0x7f0024009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a3c1f25598e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002dffb700 1 -- 172.21.15.138:0/3475834113 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0024004160 con 0x7f00300939f0 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002dffb700 1 -- 172.21.15.138:0/3475834113 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00240042c0 con 0x7f00300939f0 2024-04-27T04:34:14.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.758+0000 7f002dffb700 1 -- 172.21.15.138:0/3475834113 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0024004580 con 0x7f00300939f0 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 -- 172.21.15.138:0/3475834113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 msgr2=0x7f0030093dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 --2- 172.21.15.138:0/3475834113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f0030093dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00240097c0 tx=0x7f0024009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 -- 172.21.15.138:0/3475834113 shutdown_connections 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 --2- 172.21.15.138:0/3475834113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f0030093dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 -- 172.21.15.138:0/3475834113 >> 172.21.15.138:0/3475834113 conn(0x7f0030099b60 msgr2=0x7f0030099f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 -- 172.21.15.138:0/3475834113 shutdown_connections 2024-04-27T04:34:14.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.759+0000 7f003535e700 1 -- 172.21.15.138:0/3475834113 wait complete. 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f003535e700 1 Processor -- start 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f003535e700 1 -- start start 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f003535e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f00301bcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f003535e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301bc5f0 con 0x7f00300939f0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f002effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f00301bcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f002effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f00301bcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47726/0 (socket says 172.21.15.138:47726) 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.760+0000 7f002effd700 1 -- 172.21.15.138:0/1252061497 learned_addr learned my addr 172.21.15.138:0/1252061497 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f002effd700 1 -- 172.21.15.138:0/1252061497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00240094a0 con 0x7f00300939f0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f002effd700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f00301bcab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0024000c00 tx=0x7f00240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0024011630 con 0x7f00300939f0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0024011790 con 0x7f00300939f0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0024011920 con 0x7f00300939f0 2024-04-27T04:34:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00301a6c40 con 0x7f00300939f0 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.761+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0030091e10 con 0x7f00300939f0 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.762+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0024011a80 con 0x7f00300939f0 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.762+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0030092650 con 0x7f00300939f0 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.762+0000 7f00177fe700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f001c037780 0x7f001c039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.762+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f002404b960 con 0x7f00300939f0 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.762+0000 7f002e7fc700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f001c037780 0x7f001c039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.763+0000 7f002e7fc700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f001c037780 0x7f001c039c40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0020007fd0 tx=0x7f00200079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:14.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.767+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0024011d30 con 0x7f00300939f0 2024-04-27T04:34:14.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:14.956+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f003009cb70 con 0x7f00300939f0 2024-04-27T04:34:15.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.681+0000 7f00177fe700 1 -- 172.21.15.138:0/1252061497 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f002402d3a0 con 0x7f00300939f0 2024-04-27T04:34:15.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f001c037780 msgr2=0x7f001c039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:15.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f001c037780 0x7f001c039c40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0020007fd0 tx=0x7f00200079a0 comp rx=0 tx=0).stop 2024-04-27T04:34:15.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 msgr2=0x7f00301bcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:15.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f00301bcab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0024000c00 tx=0x7f00240047c0 comp rx=0 tx=0).stop 2024-04-27T04:34:15.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 shutdown_connections 2024-04-27T04:34:15.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f001c037780 0x7f001c039c40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:15.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 --2- 172.21.15.138:0/1252061497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00300939f0 0x7f00301bcab0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:15.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 >> 172.21.15.138:0/1252061497 conn(0x7f0030099b60 msgr2=0x7f0030088fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:15.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.684+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 shutdown_connections 2024-04-27T04:34:15.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.685+0000 7f003535e700 1 -- 172.21.15.138:0/1252061497 wait complete. 2024-04-27T04:34:15.687 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'rbd' created 2024-04-27T04:34:15.699 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-27T04:34:15.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f20458ae700 1 Processor -- start 2024-04-27T04:34:15.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f20458ae700 1 -- start start 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f20458ae700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2040094170 0x7f2040094550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f20458ae700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2040095750 con 0x7f2040094a90 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f20458ae700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204006d440 con 0x7f2040094170 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f203effd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2040094170 0x7f2040094550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f203e7fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2040094a90 0x7f2040131da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43720/0 (socket says 172.21.15.138:43720) 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.820+0000 7f203e7fc700 1 -- 172.21.15.138:0/1956321560 learned_addr learned my addr 172.21.15.138:0/1956321560 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f203effd700 1 -- 172.21.15.138:0/1956321560 >> v1:172.21.15.138:6789/0 conn(0x7f2040094a90 legacy=0x7f2040131da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f203effd700 1 -- 172.21.15.138:0/1956321560 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204006cc20 con 0x7f2040094170 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f203effd700 1 --2- 172.21.15.138:0/1956321560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f2040094550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20340097f0 tx=0x7f2034009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4aee666ba791be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f203dffb700 1 -- 172.21.15.138:0/1956321560 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2034004160 con 0x7f2040094170 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f203dffb700 1 -- 172.21.15.138:0/1956321560 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20340042c0 con 0x7f2040094170 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f203dffb700 1 -- 172.21.15.138:0/1956321560 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2034004580 con 0x7f2040094170 2024-04-27T04:34:15.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f20458ae700 1 -- 172.21.15.138:0/1956321560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 msgr2=0x7f2040094550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f20458ae700 1 --2- 172.21.15.138:0/1956321560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f2040094550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20340097f0 tx=0x7f2034009b00 comp rx=0 tx=0).stop 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f20458ae700 1 -- 172.21.15.138:0/1956321560 shutdown_connections 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f20458ae700 1 --2- 172.21.15.138:0/1956321560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f2040094550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.821+0000 7f20458ae700 1 -- 172.21.15.138:0/1956321560 >> 172.21.15.138:0/1956321560 conn(0x7f204009cde0 msgr2=0x7f204009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.822+0000 7f20458ae700 1 -- 172.21.15.138:0/1956321560 shutdown_connections 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.822+0000 7f20458ae700 1 -- 172.21.15.138:0/1956321560 wait complete. 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.822+0000 7f20458ae700 1 Processor -- start 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f20458ae700 1 -- start start 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f20458ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f20401a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f20458ae700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401a7d10 con 0x7f2040094170 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f203effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f20401a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f203effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f20401a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47744/0 (socket says 172.21.15.138:47744) 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f203effd700 1 -- 172.21.15.138:0/321434032 learned_addr learned my addr 172.21.15.138:0/321434032 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f203effd700 1 -- 172.21.15.138:0/321434032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340094a0 con 0x7f2040094170 2024-04-27T04:34:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f203effd700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f20401a6c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2034009b40 tx=0x7f2034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2034011630 con 0x7f2040094170 2024-04-27T04:34:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203401e070 con 0x7f2040094170 2024-04-27T04:34:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20340118e0 con 0x7f2040094170 2024-04-27T04:34:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401a8380 con 0x7f2040094170 2024-04-27T04:34:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.823+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401aa0d0 con 0x7f2040094170 2024-04-27T04:34:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.824+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2034011ad0 con 0x7f2040094170 2024-04-27T04:34:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.824+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20401a9400 con 0x7f2040094170 2024-04-27T04:34:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.824+0000 7f2027fff700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f202c037780 0x7f202c039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.824+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3471+0+0 (secure 0 0 0) 0x7f203404b380 con 0x7f2040094170 2024-04-27T04:34:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.825+0000 7f203e7fc700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f202c037780 0x7f202c039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.825+0000 7f203e7fc700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f202c037780 0x7f202c039c40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2030000f60 tx=0x7f2030007b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:15.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:15.829+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f203401a430 con 0x7f2040094170 2024-04-27T04:34:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.019+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2040062510 con 0x7f2040094170 2024-04-27T04:34:16.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.682+0000 7f2027fff700 1 -- 172.21.15.138:0/321434032 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f20340107f0 con 0x7f2040094170 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f202c037780 msgr2=0x7f202c039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f202c037780 0x7f202c039c40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2030000f60 tx=0x7f2030007b10 comp rx=0 tx=0).stop 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 msgr2=0x7f20401a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f20401a6c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2034009b40 tx=0x7f2034004750 comp rx=0 tx=0).stop 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 shutdown_connections 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f202c037780 0x7f202c039c40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 --2- 172.21.15.138:0/321434032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2040094170 0x7f20401a6c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 >> 172.21.15.138:0/321434032 conn(0x7f204009cde0 msgr2=0x7f2040071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 shutdown_connections 2024-04-27T04:34:16.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.686+0000 7f20458ae700 1 -- 172.21.15.138:0/321434032 wait complete. 2024-04-27T04:34:16.689 INFO:teuthology.orchestra.run.smithi138.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-27T04:34:16.701 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-27T04:34:16.701 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-27T04:34:16.702 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-27T04:34:16.702 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-27T04:34:16.702 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-27T04:34:16.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.823+0000 7f3181437700 1 Processor -- start 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.823+0000 7f3181437700 1 -- start start 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.823+0000 7f3181437700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f317c12a710 0x7f317c12aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.823+0000 7f3181437700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317c0970b0 con 0x7f317c12b030 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.823+0000 7f3181437700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c09db40 con 0x7f317c12a710 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f317affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f317c12a710 0x7f317c12aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f317a7fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f317c12b030 0x7f317c127210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43728/0 (socket says 172.21.15.138:43728) 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f317a7fc700 1 -- 172.21.15.138:0/3416284213 learned_addr learned my addr 172.21.15.138:0/3416284213 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f317affd700 1 -- 172.21.15.138:0/3416284213 >> v1:172.21.15.138:6789/0 conn(0x7f317c12b030 legacy=0x7f317c127210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f317affd700 1 -- 172.21.15.138:0/3416284213 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c096960 con 0x7f317c12a710 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f3179ffb700 1 -- 172.21.15.138:0/3416284213 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538123837 0 0) 0x7f317c0970b0 con 0x7f317c12b030 2024-04-27T04:34:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f317affd700 1 --2- 172.21.15.138:0/3416284213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c12aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31700097c0 tx=0x7f3170009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbffb4c6b1a3f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f3179ffb700 1 -- 172.21.15.138:0/3416284213 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3170004160 con 0x7f317c12a710 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f3179ffb700 1 -- 172.21.15.138:0/3416284213 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31700042c0 con 0x7f317c12a710 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.824+0000 7f3179ffb700 1 -- 172.21.15.138:0/3416284213 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3170004580 con 0x7f317c12a710 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 -- 172.21.15.138:0/3416284213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 msgr2=0x7f317c12aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 --2- 172.21.15.138:0/3416284213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c12aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31700097c0 tx=0x7f3170009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 -- 172.21.15.138:0/3416284213 shutdown_connections 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 --2- 172.21.15.138:0/3416284213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c12aaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 -- 172.21.15.138:0/3416284213 >> 172.21.15.138:0/3416284213 conn(0x7f317c09d020 msgr2=0x7f317c09d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 -- 172.21.15.138:0/3416284213 shutdown_connections 2024-04-27T04:34:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.825+0000 7f3181437700 1 -- 172.21.15.138:0/3416284213 wait complete. 2024-04-27T04:34:16.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f3181437700 1 Processor -- start 2024-04-27T04:34:16.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f3181437700 1 -- start start 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f3181437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c1237b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f3181437700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c126170 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f317affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c1237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f317affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c1237b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47764/0 (socket says 172.21.15.138:47764) 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f317affd700 1 -- 172.21.15.138:0/920900997 learned_addr learned my addr 172.21.15.138:0/920900997 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.826+0000 7f317affd700 1 -- 172.21.15.138:0/920900997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31700094a0 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f317affd700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c1237b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3170009b10 tx=0x7f31700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3170011630 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3170011790 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317c124fd0 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3170011a70 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317c125550 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.827+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3170011bd0 con 0x7f317c12a710 2024-04-27T04:34:16.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.828+0000 7f31637fe700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f31680377b0 0x7f3168039c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:16.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.828+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f317004b810 con 0x7f317c12a710 2024-04-27T04:34:16.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.828+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317c093290 con 0x7f317c12a710 2024-04-27T04:34:16.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.828+0000 7f317a7fc700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f31680377b0 0x7f3168039c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:16.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.828+0000 7f317a7fc700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f31680377b0 0x7f3168039c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f316c007fd0 tx=0x7f316c0079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:16.832+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f317c093290 con 0x7f317c12a710 2024-04-27T04:34:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.077+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f317c093290 con 0x7f317c12a710 2024-04-27T04:34:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.078+0000 7f31637fe700 1 -- 172.21.15.138:0/920900997 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+86131 (secure 0 0 0) 0x7f317c093290 con 0x7f317c12a710 2024-04-27T04:34:17.081 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.080+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f31680377b0 msgr2=0x7f3168039c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.080+0000 7f3181437700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f31680377b0 0x7f3168039c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f316c007fd0 tx=0x7f316c0079a0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.080+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 msgr2=0x7f317c1237b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.080+0000 7f3181437700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c1237b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3170009b10 tx=0x7f31700047c0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.081+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 shutdown_connections 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.081+0000 7f3181437700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f31680377b0 0x7f3168039c70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.081+0000 7f3181437700 1 --2- 172.21.15.138:0/920900997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f317c12a710 0x7f317c1237b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.081+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 >> 172.21.15.138:0/920900997 conn(0x7f317c09d020 msgr2=0x7f317c093640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.081+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 shutdown_connections 2024-04-27T04:34:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.081+0000 7f3181437700 1 -- 172.21.15.138:0/920900997 wait complete. 2024-04-27T04:34:17.093 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":41511},{"type":"v1","addr":"172.21.15.191:6825","nonce":41511}]},"active_addr":"172.21.15.191:6825/41511","active_change":"2024-04-27T04:34:05.086104+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":3565276384}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":483276280}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":356768146}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":1112445856}]}]} 2024-04-27T04:34:17.095 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-27T04:34:17.096 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-27T04:34:17.096 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:34:17.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f97421b0700 1 Processor -- start 2024-04-27T04:34:17.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f97421b0700 1 -- start start 2024-04-27T04:34:17.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f97421b0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f973c130c60 0x7f973c133050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f97421b0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c099a20 con 0x7f973c097ae0 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f97421b0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c098cb0 con 0x7f973c130c60 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f973affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f973c130c60 0x7f973c133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f973b7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f973c097ae0 0x7f973c097ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43744/0 (socket says 172.21.15.138:43744) 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.214+0000 7f973b7fe700 1 -- 172.21.15.138:0/446966508 learned_addr learned my addr 172.21.15.138:0/446966508 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f973affd700 1 -- 172.21.15.138:0/446966508 >> v1:172.21.15.138:6789/0 conn(0x7f973c097ae0 legacy=0x7f973c097ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f973affd700 1 -- 172.21.15.138:0/446966508 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c099f60 con 0x7f973c130c60 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f973affd700 1 --2- 172.21.15.138:0/446966508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c130c60 0x7f973c133050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97300097c0 tx=0x7f9730009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359f4b42038f4161 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f973a7fc700 1 -- 172.21.15.138:0/446966508 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9730004160 con 0x7f973c130c60 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f973a7fc700 1 -- 172.21.15.138:0/446966508 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97300042c0 con 0x7f973c130c60 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f973a7fc700 1 -- 172.21.15.138:0/446966508 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9730004580 con 0x7f973c130c60 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 -- 172.21.15.138:0/446966508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c130c60 msgr2=0x7f973c133050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 --2- 172.21.15.138:0/446966508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c130c60 0x7f973c133050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97300097c0 tx=0x7f9730009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 -- 172.21.15.138:0/446966508 shutdown_connections 2024-04-27T04:34:17.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 --2- 172.21.15.138:0/446966508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c130c60 0x7f973c133050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 -- 172.21.15.138:0/446966508 >> 172.21.15.138:0/446966508 conn(0x7f973c09e0e0 msgr2=0x7f973c09e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 -- 172.21.15.138:0/446966508 shutdown_connections 2024-04-27T04:34:17.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.215+0000 7f97421b0700 1 -- 172.21.15.138:0/446966508 wait complete. 2024-04-27T04:34:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.216+0000 7f97421b0700 1 Processor -- start 2024-04-27T04:34:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.216+0000 7f97421b0700 1 -- start start 2024-04-27T04:34:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f97421b0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 0x7f973c12d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f97421b0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c133360 con 0x7f973c097ae0 2024-04-27T04:34:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f973b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 0x7f973c12d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f973b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 0x7f973c12d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47768/0 (socket says 172.21.15.138:47768) 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f973b7fe700 1 -- 172.21.15.138:0/3815043911 learned_addr learned my addr 172.21.15.138:0/3815043911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f973b7fe700 1 -- 172.21.15.138:0/3815043911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300094a0 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f973b7fe700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 0x7f973c12d1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c410 tx=0x7f972c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f972c005ca0 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c005e00 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f972c0049c0 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c092260 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.217+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c08d610 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.218+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f972c004b20 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.218+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c08eeb0 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.218+0000 7f9723fff700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9728037790 0x7f9728039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.218+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f972c049840 con 0x7f973c097ae0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.219+0000 7f973affd700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9728037790 0x7f9728039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.219+0000 7f973affd700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9728037790 0x7f9728039c50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b10 tx=0x7f9730008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.223+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f973c08eeb0 con 0x7f973c097ae0 2024-04-27T04:34:17.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.414+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f973c08eeb0 con 0x7f973c097ae0 2024-04-27T04:34:17.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.415+0000 7f9723fff700 1 -- 172.21.15.138:0/3815043911 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+9476 (secure 0 0 0) 0x7f973c08eeb0 con 0x7f973c097ae0 2024-04-27T04:34:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:17.416 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":15,"fsid":"512e09fb-564f-4fb6-aa0b-f541dbeb017d","created":"2024-04-27T04:33:59.121696+0000","modified":"2024-04-27T04:34:16.680959+0000","last_up_change":"2024-04-27T04:34:08.432381+0000","last_in_change":"2024-04-27T04:34:00.320575+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:34:09.130881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T04:34:14.973814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74a1ee1e-2f1c-4eaa-860a-ad1e52733bde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":39496},{"type":"v1","addr":"172.21.15.138:6801","nonce":39496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":39496},{"type":"v1","addr":"172.21.15.138:6803","nonce":39496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":39496},{"type":"v1","addr":"172.21.15.138:6807","nonce":39496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":39496},{"type":"v1","addr":"172.21.15.138:6805","nonce":39496}]},"public_addr":"172.21.15.138:6801/39496","cluster_addr":"172.21.15.138:6803/39496","heartbeat_back_addr":"172.21.15.138:6807/39496","heartbeat_front_addr":"172.21.15.138:6805/39496","state":["exists","up"]},{"osd":1,"uuid":"d22f26e1-ca4f-4271-994f-b7011b3f030c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":41886},{"type":"v1","addr":"172.21.15.191:6809","nonce":41886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":41886},{"type":"v1","addr":"172.21.15.191:6811","nonce":41886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":41886},{"type":"v1","addr":"172.21.15.191:6815","nonce":41886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":41886},{"type":"v1","addr":"172.21.15.191:6813","nonce":41886}]},"public_addr":"172.21.15.191:6809/41886","cluster_addr":"172.21.15.191:6811/41886","heartbeat_back_addr":"172.21.15.191:6815/41886","heartbeat_front_addr":"172.21.15.191:6813/41886","state":["exists","up"]},{"osd":2,"uuid":"699aaac0-41d4-47af-ae91-100b87b28b45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":41888},{"type":"v1","addr":"172.21.15.191:6817","nonce":41888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":41888},{"type":"v1","addr":"172.21.15.191:6819","nonce":41888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":41888},{"type":"v1","addr":"172.21.15.191:6823","nonce":41888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":41888},{"type":"v1","addr":"172.21.15.191:6821","nonce":41888}]},"public_addr":"172.21.15.191:6817/41888","cluster_addr":"172.21.15.191:6819/41888","heartbeat_back_addr":"172.21.15.191:6823/41888","heartbeat_front_addr":"172.21.15.191:6821/41888","state":["exists","up"]},{"osd":3,"uuid":"cecc3ff7-644a-480d-ba0b-ac744a496062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":41887},{"type":"v1","addr":"172.21.15.191:6801","nonce":41887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":41887},{"type":"v1","addr":"172.21.15.191:6803","nonce":41887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":41887},{"type":"v1","addr":"172.21.15.191:6807","nonce":41887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":41887},{"type":"v1","addr":"172.21.15.191:6805","nonce":41887}]},"public_addr":"172.21.15.191:6801/41887","cluster_addr":"172.21.15.191:6803/41887","heartbeat_back_addr":"172.21.15.191:6807/41887","heartbeat_front_addr":"172.21.15.191:6805/41887","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.420983+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.449423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.499228+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.424812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9728037790 msgr2=0x7f9728039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9728037790 0x7f9728039c50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b10 tx=0x7f9730008030 comp rx=0 tx=0).stop 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 msgr2=0x7f973c12d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 0x7f973c12d1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c410 tx=0x7f972c00c7d0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 shutdown_connections 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9728037790 0x7f9728039c50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 --2- 172.21.15.138:0/3815043911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c097ae0 0x7f973c12d1c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 >> 172.21.15.138:0/3815043911 conn(0x7f973c09e0e0 msgr2=0x7f973c094750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 shutdown_connections 2024-04-27T04:34:17.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.417+0000 7f97421b0700 1 -- 172.21.15.138:0/3815043911 wait complete. 2024-04-27T04:34:17.430 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-27T04:34:17.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T04:34:17.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9ff8de700 1 Processor -- start 2024-04-27T04:34:17.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9ff8de700 1 -- start start 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9ff8de700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9f812a080 0x7ff9f8130a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9ff8de700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f8097a00 con 0x7ff9f8129590 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9ff8de700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f8067300 con 0x7ff9f812a080 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9fce79700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9f812a080 0x7ff9f8130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9fd67a700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff9f8129590 0x7ff9f8129970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43756/0 (socket says 172.21.15.138:43756) 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9fce79700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9f812a080 0x7ff9f8130a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47774/0 (socket says 172.21.15.138:47774) 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.548+0000 7ff9fd67a700 1 -- 172.21.15.138:0/1709153069 learned_addr learned my addr 172.21.15.138:0/1709153069 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9fce79700 1 -- 172.21.15.138:0/1709153069 >> v1:172.21.15.138:6789/0 conn(0x7ff9f8129590 legacy=0x7ff9f8129970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9fce79700 1 -- 172.21.15.138:0/1709153069 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8097f40 con 0x7ff9f812a080 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9effff700 1 -- 172.21.15.138:0/1709153069 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828280354 0 0) 0x7ff9f8097a00 con 0x7ff9f8129590 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9fce79700 1 --2- 172.21.15.138:0/1709153069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f812a080 0x7ff9f8130a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80097c0 tx=0x7ff9e8009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4898fb0ac4b04199 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9effff700 1 -- 172.21.15.138:0/1709153069 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8004160 con 0x7ff9f812a080 2024-04-27T04:34:17.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9effff700 1 -- 172.21.15.138:0/1709153069 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e80042c0 con 0x7ff9f812a080 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9effff700 1 -- 172.21.15.138:0/1709153069 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8004580 con 0x7ff9f812a080 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9ff8de700 1 -- 172.21.15.138:0/1709153069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f812a080 msgr2=0x7ff9f8130a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9ff8de700 1 --2- 172.21.15.138:0/1709153069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f812a080 0x7ff9f8130a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80097c0 tx=0x7ff9e8009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9ff8de700 1 -- 172.21.15.138:0/1709153069 shutdown_connections 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9ff8de700 1 --2- 172.21.15.138:0/1709153069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f812a080 0x7ff9f8130a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9ff8de700 1 -- 172.21.15.138:0/1709153069 >> 172.21.15.138:0/1709153069 conn(0x7ff9f809be70 msgr2=0x7ff9f809c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.549+0000 7ff9ff8de700 1 -- 172.21.15.138:0/1709153069 shutdown_connections 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.550+0000 7ff9ff8de700 1 -- 172.21.15.138:0/1709153069 wait complete. 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.550+0000 7ff9ff8de700 1 Processor -- start 2024-04-27T04:34:17.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.550+0000 7ff9ff8de700 1 -- start start 2024-04-27T04:34:17.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9ff8de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 0x7ff9f8123840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9ff8de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8126eb0 con 0x7ff9f8129590 2024-04-27T04:34:17.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9fd67a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 0x7ff9f8123840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9fd67a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 0x7ff9f8123840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47782/0 (socket says 172.21.15.138:47782) 2024-04-27T04:34:17.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9fd67a700 1 -- 172.21.15.138:0/2251196423 learned_addr learned my addr 172.21.15.138:0/2251196423 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9fd67a700 1 -- 172.21.15.138:0/2251196423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80094a0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9fd67a700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 0x7ff9f8123840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400c4a0 tx=0x7ff9f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9f4004bb0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f4004d50 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f81256e0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.551+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9f40070d0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.552+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f812c4a0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.552+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff9f40072c0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.552+0000 7ff9edffb700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ff9dc037790 0x7ff9dc039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.552+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7ff9f4049cb0 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.552+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f8062510 con 0x7ff9f8129590 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.552+0000 7ff9fce79700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ff9dc037790 0x7ff9dc039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.553+0000 7ff9fce79700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ff9dc037790 0x7ff9dc039c50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009b10 tx=0x7ff9e8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.557+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ff9f40102c0 con 0x7ff9f8129590 2024-04-27T04:34:17.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.750+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9f8066af0 con 0x7ff9f8129590 2024-04-27T04:34:17.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.750+0000 7ff9edffb700 1 -- 172.21.15.138:0/2251196423 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+9463 (secure 0 0 0) 0x7ff9f4011360 con 0x7ff9f8129590 2024-04-27T04:34:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:17.752 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"fsid":"512e09fb-564f-4fb6-aa0b-f541dbeb017d","created":"2024-04-27T04:33:59.121696+0000","modified":"2024-04-27T04:34:17.684907+0000","last_up_change":"2024-04-27T04:34:08.432381+0000","last_in_change":"2024-04-27T04:34:00.320575+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T04:34:09.130881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T04:34:14.973814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74a1ee1e-2f1c-4eaa-860a-ad1e52733bde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":39496},{"type":"v1","addr":"172.21.15.138:6801","nonce":39496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":39496},{"type":"v1","addr":"172.21.15.138:6803","nonce":39496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":39496},{"type":"v1","addr":"172.21.15.138:6807","nonce":39496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":39496},{"type":"v1","addr":"172.21.15.138:6805","nonce":39496}]},"public_addr":"172.21.15.138:6801/39496","cluster_addr":"172.21.15.138:6803/39496","heartbeat_back_addr":"172.21.15.138:6807/39496","heartbeat_front_addr":"172.21.15.138:6805/39496","state":["exists","up"]},{"osd":1,"uuid":"d22f26e1-ca4f-4271-994f-b7011b3f030c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":41886},{"type":"v1","addr":"172.21.15.191:6809","nonce":41886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":41886},{"type":"v1","addr":"172.21.15.191:6811","nonce":41886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":41886},{"type":"v1","addr":"172.21.15.191:6815","nonce":41886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":41886},{"type":"v1","addr":"172.21.15.191:6813","nonce":41886}]},"public_addr":"172.21.15.191:6809/41886","cluster_addr":"172.21.15.191:6811/41886","heartbeat_back_addr":"172.21.15.191:6815/41886","heartbeat_front_addr":"172.21.15.191:6813/41886","state":["exists","up"]},{"osd":2,"uuid":"699aaac0-41d4-47af-ae91-100b87b28b45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":41888},{"type":"v1","addr":"172.21.15.191:6817","nonce":41888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":41888},{"type":"v1","addr":"172.21.15.191:6819","nonce":41888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":41888},{"type":"v1","addr":"172.21.15.191:6823","nonce":41888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":41888},{"type":"v1","addr":"172.21.15.191:6821","nonce":41888}]},"public_addr":"172.21.15.191:6817/41888","cluster_addr":"172.21.15.191:6819/41888","heartbeat_back_addr":"172.21.15.191:6823/41888","heartbeat_front_addr":"172.21.15.191:6821/41888","state":["exists","up"]},{"osd":3,"uuid":"cecc3ff7-644a-480d-ba0b-ac744a496062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":41887},{"type":"v1","addr":"172.21.15.191:6801","nonce":41887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":41887},{"type":"v1","addr":"172.21.15.191:6803","nonce":41887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":41887},{"type":"v1","addr":"172.21.15.191:6807","nonce":41887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":41887},{"type":"v1","addr":"172.21.15.191:6805","nonce":41887}]},"public_addr":"172.21.15.191:6801/41887","cluster_addr":"172.21.15.191:6803/41887","heartbeat_back_addr":"172.21.15.191:6807/41887","heartbeat_front_addr":"172.21.15.191:6805/41887","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.420983+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.449423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.499228+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T04:34:05.424812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T04:34:17.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ff9dc037790 msgr2=0x7ff9dc039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ff9dc037790 0x7ff9dc039c50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009b10 tx=0x7ff9e8008030 comp rx=0 tx=0).stop 2024-04-27T04:34:17.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 msgr2=0x7ff9f8123840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 0x7ff9f8123840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400c4a0 tx=0x7ff9f400c860 comp rx=0 tx=0).stop 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 shutdown_connections 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ff9dc037790 0x7ff9dc039c50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 --2- 172.21.15.138:0/2251196423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f8129590 0x7ff9f8123840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 >> 172.21.15.138:0/2251196423 conn(0x7ff9f809be70 msgr2=0x7ff9f8099020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.752+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 shutdown_connections 2024-04-27T04:34:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.753+0000 7ff9ff8de700 1 -- 172.21.15.138:0/2251196423 wait complete. 2024-04-27T04:34:17.766 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-27T04:34:17.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-27T04:34:17.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-27T04:34:17.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-27T04:34:17.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.895+0000 7f4a0d038700 1 Processor -- start 2024-04-27T04:34:17.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.895+0000 7f4a0d038700 1 -- start start 2024-04-27T04:34:17.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.896+0000 7f4a0d038700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a0806ead0 0x7f4a081331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.896+0000 7f4a0d038700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a08072810 con 0x7f4a0812e1e0 2024-04-27T04:34:17.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.896+0000 7f4a0d038700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a08072bc0 con 0x7f4a0806ead0 2024-04-27T04:34:17.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.896+0000 7f4a06d9d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4a0812e1e0 0x7f4a0812e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43768/0 (socket says 172.21.15.138:43768) 2024-04-27T04:34:17.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.896+0000 7f4a0659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a0806ead0 0x7f4a081331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.896+0000 7f4a06d9d700 1 -- 172.21.15.138:0/28113548 learned_addr learned my addr 172.21.15.138:0/28113548 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:17.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a0659c700 1 -- 172.21.15.138:0/28113548 >> v1:172.21.15.138:6789/0 conn(0x7f4a0812e1e0 legacy=0x7f4a0812e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a0659c700 1 -- 172.21.15.138:0/28113548 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0806f9e0 con 0x7f4a0806ead0 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a05d9b700 1 -- 172.21.15.138:0/28113548 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746697655 0 0) 0x7f49f8001110 con 0x7f4a0812e1e0 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a0659c700 1 --2- 172.21.15.138:0/28113548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a081331b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097c0 tx=0x7f49fc009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c245f8f3fa7333 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a05d9b700 1 -- 172.21.15.138:0/28113548 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49fc004160 con 0x7f4a0806ead0 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a05d9b700 1 -- 172.21.15.138:0/28113548 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc0042c0 con 0x7f4a0806ead0 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.897+0000 7f4a05d9b700 1 -- 172.21.15.138:0/28113548 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49fc004580 con 0x7f4a0806ead0 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 -- 172.21.15.138:0/28113548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 msgr2=0x7f4a081331b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 --2- 172.21.15.138:0/28113548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a081331b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097c0 tx=0x7f49fc009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 -- 172.21.15.138:0/28113548 shutdown_connections 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 --2- 172.21.15.138:0/28113548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a081331b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 -- 172.21.15.138:0/28113548 >> 172.21.15.138:0/28113548 conn(0x7f4a0809c120 msgr2=0x7f4a0809c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 -- 172.21.15.138:0/28113548 shutdown_connections 2024-04-27T04:34:17.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.898+0000 7f4a0d038700 1 -- 172.21.15.138:0/28113548 wait complete. 2024-04-27T04:34:17.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.899+0000 7f4a0d038700 1 Processor -- start 2024-04-27T04:34:17.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.899+0000 7f4a0d038700 1 -- start start 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a0d038700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a0808dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a0d038700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a08094510 con 0x7f4a0806ead0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a06d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a0808dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a06d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a0808dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47800/0 (socket says 172.21.15.138:47800) 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a06d9d700 1 -- 172.21.15.138:0/3589983513 learned_addr learned my addr 172.21.15.138:0/3589983513 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a06d9d700 1 -- 172.21.15.138:0/3589983513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0094a0 con 0x7f4a0806ead0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.900+0000 7f4a06d9d700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a0808dfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49f800c4a0 tx=0x7f49f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.901+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49f8004bb0 con 0x7f4a0806ead0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.901+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f8004d10 con 0x7f4a0806ead0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.901+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49f80070d0 con 0x7f4a0806ead0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.901+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a08091270 con 0x7f4a0806ead0 2024-04-27T04:34:17.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.901+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0808fce0 con 0x7f4a0806ead0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a08091400 con 0x7f4a0806ead0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f49f80072e0 con 0x7f4a0806ead0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f49ef7fe700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f49f40377e0 0x7f49f4039ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f49f8049b90 con 0x7f4a0806ead0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f49ef7fe700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x7f49f403cf00 0x7f49f403f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49f403fa10 con 0x7f49f403cf00 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f804e1c0 con 0x7f4a0806ead0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f4a0659c700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f49f40377e0 0x7f49f4039ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.902+0000 7f4a0759e700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x7f49f403cf00 0x7f49f403f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.903+0000 7f4a0659c700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f49f40377e0 0x7f49f4039ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0090b0 tx=0x7f49fc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.903+0000 7f4a0759e700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x7f49f403cf00 0x7f49f403f340 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.904+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f49f403fa10 con 0x7f49f403cf00 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.904+0000 7ffafa870700 1 Processor -- start 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.904+0000 7ffafa870700 1 -- start start 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffafa870700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffaf4130c60 0x7ffaf4133050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffafa870700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf4099a20 con 0x7ffaf4097ae0 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffafa870700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf4098cb0 con 0x7ffaf4130c60 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffaf37fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffaf4130c60 0x7ffaf4133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffaf37fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffaf4130c60 0x7ffaf4133050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47804/0 (socket says 172.21.15.138:47804) 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffaf37fe700 1 -- 172.21.15.138:0/3995910479 learned_addr learned my addr 172.21.15.138:0/3995910479 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffaf37fe700 1 -- 172.21.15.138:0/3995910479 >> v1:172.21.15.138:6789/0 conn(0x7ffaf4097ae0 legacy=0x7ffaf4097ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffaf37fe700 1 -- 172.21.15.138:0/3995910479 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4099f60 con 0x7ffaf4130c60 2024-04-27T04:34:17.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.905+0000 7ffaf37fe700 1 --2- 172.21.15.138:0/3995910479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4130c60 0x7ffaf4133050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009870 tx=0x7ffae8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab84fb44a112735 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffaf2ffd700 1 -- 172.21.15.138:0/3995910479 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae8004160 con 0x7ffaf4130c60 2024-04-27T04:34:17.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffaf2ffd700 1 -- 172.21.15.138:0/3995910479 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae80042c0 con 0x7ffaf4130c60 2024-04-27T04:34:17.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffaf2ffd700 1 -- 172.21.15.138:0/3995910479 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae8004580 con 0x7ffaf4130c60 2024-04-27T04:34:17.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 -- 172.21.15.138:0/3995910479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4130c60 msgr2=0x7ffaf4133050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 --2- 172.21.15.138:0/3995910479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4130c60 0x7ffaf4133050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009870 tx=0x7ffae8009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:17.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 -- 172.21.15.138:0/3995910479 shutdown_connections 2024-04-27T04:34:17.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 --2- 172.21.15.138:0/3995910479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4130c60 0x7ffaf4133050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 -- 172.21.15.138:0/3995910479 >> 172.21.15.138:0/3995910479 conn(0x7ffaf409e0e0 msgr2=0x7ffaf409e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 -- 172.21.15.138:0/3995910479 shutdown_connections 2024-04-27T04:34:17.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.907+0000 7ffafa870700 1 -- 172.21.15.138:0/3995910479 wait complete. 2024-04-27T04:34:17.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.908+0000 7ffafa870700 1 Processor -- start 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.908+0000 7ffafa870700 1 -- start start 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffafa870700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 0x7ffaf41c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffafa870700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf412e5b0 con 0x7ffaf4097ae0 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 0x7ffaf41c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 0x7ffaf41c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47818/0 (socket says 172.21.15.138:47818) 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffaf3fff700 1 -- 172.21.15.138:0/3560672417 learned_addr learned my addr 172.21.15.138:0/3560672417 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffaf3fff700 1 -- 172.21.15.138:0/3560672417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae8009550 con 0x7ffaf4097ae0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.909+0000 7ffaf3fff700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 0x7ffaf41c6500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffae400c410 tx=0x7ffae400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.910+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae4005200 con 0x7ffaf4097ae0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.910+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae4005380 con 0x7ffaf4097ae0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.910+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf4126550 con 0x7ffaf4097ae0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.910+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae4006860 con 0x7ffaf4097ae0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.910+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf4126b40 con 0x7ffaf4097ae0 2024-04-27T04:34:17.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffaf40985d0 con 0x7ffaf4097ae0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ffae4003680 con 0x7ffaf4097ae0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffaf0ff9700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ffae0037790 0x7ffae0039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7ffae40533a0 con 0x7ffaf4097ae0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffaf0ff9700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x7ffae003ceb0 0x7ffae003f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffae003f9c0 con 0x7ffae003ceb0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.911+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffae40536a0 con 0x7ffaf4097ae0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.912+0000 7ffaf37fe700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ffae0037790 0x7ffae0039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.912+0000 7ffaf8e0d700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x7ffae003ceb0 0x7ffae003f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.912+0000 7ffaf37fe700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ffae0037790 0x7ffae0039c50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009bc0 tx=0x7ffae8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.912+0000 7ffaf8e0d700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x7ffae003ceb0 0x7ffae003f2f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.913+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ffae003f9c0 con 0x7ffae003ceb0 2024-04-27T04:34:17.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.914+0000 7f363180f700 1 Processor -- start 2024-04-27T04:34:17.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.914+0000 7f363180f700 1 -- start start 2024-04-27T04:34:17.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f363180f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f362c12d010 0x7f362c12f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f363180f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c129d00 con 0x7f362c12a250 2024-04-27T04:34:17.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f363180f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c095900 con 0x7f362c12d010 2024-04-27T04:34:17.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f362a7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f362c12d010 0x7f362c12f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f362a7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f362c12d010 0x7f362c12f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47826/0 (socket says 172.21.15.138:47826) 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f362a7fc700 1 -- 172.21.15.138:0/250824281 learned_addr learned my addr 172.21.15.138:0/250824281 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f362a7fc700 1 -- 172.21.15.138:0/250824281 >> v1:172.21.15.138:6789/0 conn(0x7f362c12a250 legacy=0x7f362c12a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f362a7fc700 1 -- 172.21.15.138:0/250824281 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c095c50 con 0x7f362c12d010 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f362a7fc700 1 --2- 172.21.15.138:0/250824281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12d010 0x7f362c12f400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f362000bb10 tx=0x7f362000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e33235d860983c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.915+0000 7f3629ffb700 1 -- 172.21.15.138:0/250824281 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f362000db50 con 0x7f362c12d010 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f3629ffb700 1 -- 172.21.15.138:0/250824281 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362000dcb0 con 0x7f362c12d010 2024-04-27T04:34:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f3629ffb700 1 -- 172.21.15.138:0/250824281 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3620004030 con 0x7f362c12d010 2024-04-27T04:34:17.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f363180f700 1 -- 172.21.15.138:0/250824281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12d010 msgr2=0x7f362c12f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f363180f700 1 --2- 172.21.15.138:0/250824281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12d010 0x7f362c12f400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f362000bb10 tx=0x7f362000be20 comp rx=0 tx=0).stop 2024-04-27T04:34:17.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f363180f700 1 -- 172.21.15.138:0/250824281 shutdown_connections 2024-04-27T04:34:17.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f363180f700 1 --2- 172.21.15.138:0/250824281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12d010 0x7f362c12f400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.916+0000 7f363180f700 1 -- 172.21.15.138:0/250824281 >> 172.21.15.138:0/250824281 conn(0x7f362c09a870 msgr2=0x7f362c09ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.917+0000 7f363180f700 1 -- 172.21.15.138:0/250824281 shutdown_connections 2024-04-27T04:34:17.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.917+0000 7f363180f700 1 -- 172.21.15.138:0/250824281 wait complete. 2024-04-27T04:34:17.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.918+0000 7f363180f700 1 Processor -- start 2024-04-27T04:34:17.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.918+0000 7f363180f700 1 -- start start 2024-04-27T04:34:17.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f363180f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 0x7f362c088fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f363180f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c08ce70 con 0x7f362c12a250 2024-04-27T04:34:17.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f362affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 0x7f362c088fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f362affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 0x7f362c088fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47842/0 (socket says 172.21.15.138:47842) 2024-04-27T04:34:17.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f362affd700 1 -- 172.21.15.138:0/318110857 learned_addr learned my addr 172.21.15.138:0/318110857 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f362affd700 1 -- 172.21.15.138:0/318110857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362000b7f0 con 0x7f362c12a250 2024-04-27T04:34:17.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f362affd700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 0x7f362c088fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c460 tx=0x7f361c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361c005200 con 0x7f362c12a250 2024-04-27T04:34:17.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c005360 con 0x7f362c12a250 2024-04-27T04:34:17.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.919+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361c003680 con 0x7f362c12a250 2024-04-27T04:34:17.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.920+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c08b6e0 con 0x7f362c12a250 2024-04-27T04:34:17.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.920+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c092d30 con 0x7f362c12a250 2024-04-27T04:34:17.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.921+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f362c096070 con 0x7f362c12a250 2024-04-27T04:34:17.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.921+0000 7f8acfba1700 1 Processor -- start 2024-04-27T04:34:17.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.921+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f361c0037e0 con 0x7f362c12a250 2024-04-27T04:34:17.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f3613fff700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f3618037790 0x7f3618039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f361c04d120 con 0x7f362c12a250 2024-04-27T04:34:17.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f3613fff700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x7f361803ceb0 0x7f361803f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f361803f9c0 con 0x7f361803ceb0 2024-04-27T04:34:17.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f361c04d440 con 0x7f362c12a250 2024-04-27T04:34:17.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acfba1700 1 -- start start 2024-04-27T04:34:17.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acfba1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ac812a080 0x7f8ac8130a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acfba1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac8097a00 con 0x7f8ac8129590 2024-04-27T04:34:17.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acfba1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac8067300 con 0x7f8ac812a080 2024-04-27T04:34:17.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f362b7fe700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x7f361803ceb0 0x7f361803f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f362a7fc700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f3618037790 0x7f3618039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acd13c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ac812a080 0x7f8ac8130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acd13c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ac812a080 0x7f8ac8130a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47844/0 (socket says 172.21.15.138:47844) 2024-04-27T04:34:17.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acd93d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8ac8129590 0x7f8ac8129970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43800/0 (socket says 172.21.15.138:43800) 2024-04-27T04:34:17.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.922+0000 7f8acd13c700 1 -- 172.21.15.138:0/2912362210 learned_addr learned my addr 172.21.15.138:0/2912362210 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f362b7fe700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x7f361803ceb0 0x7f361803f2f0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acc93b700 1 -- 172.21.15.138:0/2912362210 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180188168 0 0) 0x7f8ac8097a00 con 0x7f8ac8129590 2024-04-27T04:34:17.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acc93b700 1 -- 172.21.15.138:0/2912362210 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab40035b0 con 0x7f8ac8129590 2024-04-27T04:34:17.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f362a7fc700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f3618037790 0x7f3618039c50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f362000be60 tx=0x7f3620014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a08090260 con 0x7f49f403cf00 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acd13c700 1 -- 172.21.15.138:0/2912362210 >> v1:172.21.15.138:6789/0 conn(0x7f8ac8129590 legacy=0x7f8ac8129970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acd13c700 1 -- 172.21.15.138:0/2912362210 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8097f40 con 0x7f8ac812a080 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acd13c700 1 --2- 172.21.15.138:0/2912362210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac812a080 0x7f8ac8130a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009870 tx=0x7f8ac4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9927f614387c0812 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acc93b700 1 -- 172.21.15.138:0/2912362210 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ac4004160 con 0x7f8ac812a080 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acc93b700 1 -- 172.21.15.138:0/2912362210 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac40042c0 con 0x7f8ac812a080 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f8acc93b700 1 -- 172.21.15.138:0/2912362210 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ac4004580 con 0x7f8ac812a080 2024-04-27T04:34:17.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f49ef7fe700 1 -- 172.21.15.138:0/3589983513 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a00011040 con 0x7f49f403cf00 2024-04-27T04:34:17.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.923+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f361803f9c0 con 0x7f361803ceb0 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x7f49f403cf00 msgr2=0x7f49f403f340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x7f49f403cf00 0x7f49f403f340 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f49f40377e0 msgr2=0x7f49f4039ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f49f40377e0 0x7f49f4039ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0090b0 tx=0x7f49fc008030 comp rx=0 tx=0).stop 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 msgr2=0x7f4a0808dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a0808dfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49f800c4a0 tx=0x7f49f800c860 comp rx=0 tx=0).stop 2024-04-27T04:34:17.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 shutdown_connections 2024-04-27T04:34:17.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x7f49f403cf00 0x7f49f403f340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0806ead0 0x7f4a0808dfb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 --2- 172.21.15.138:0/3589983513 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f49f40377e0 0x7f49f4039ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 >> 172.21.15.138:0/3589983513 conn(0x7f4a0809c120 msgr2=0x7f4a080972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 shutdown_connections 2024-04-27T04:34:17.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f4a0d038700 1 -- 172.21.15.138:0/3589983513 wait complete. 2024-04-27T04:34:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f8acfba1700 1 -- 172.21.15.138:0/2912362210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac812a080 msgr2=0x7f8ac8130a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f8acfba1700 1 --2- 172.21.15.138:0/2912362210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac812a080 0x7f8ac8130a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009870 tx=0x7f8ac4009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f8acfba1700 1 -- 172.21.15.138:0/2912362210 shutdown_connections 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f8acfba1700 1 --2- 172.21.15.138:0/2912362210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac812a080 0x7f8ac8130a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.924+0000 7f8acfba1700 1 -- 172.21.15.138:0/2912362210 >> 172.21.15.138:0/2912362210 conn(0x7f8ac809be70 msgr2=0x7f8ac809c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.925+0000 7f8acfba1700 1 -- 172.21.15.138:0/2912362210 shutdown_connections 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.925+0000 7f8acfba1700 1 -- 172.21.15.138:0/2912362210 wait complete. 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.925+0000 7f8acfba1700 1 Processor -- start 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.925+0000 7f8acfba1700 1 -- start start 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acfba1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 0x7f8ac81c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acfba1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac806dc80 con 0x7f8ac8129590 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acd93d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 0x7f8ac81c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acd93d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 0x7f8ac81c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47856/0 (socket says 172.21.15.138:47856) 2024-04-27T04:34:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acd93d700 1 -- 172.21.15.138:0/1602992969 learned_addr learned my addr 172.21.15.138:0/1602992969 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acd93d700 1 -- 172.21.15.138:0/1602992969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4009550 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8acd93d700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 0x7f8ac81c21b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000cb60 tx=0x7f8ac000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.926+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ac000aa60 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.927+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac000abc0 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.927+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ac000ae60 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.927+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac806f100 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.927+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac806ee20 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ac806e9b0 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8ac0007070 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8abe7fc700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f8aac037790 0x7f8aac039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f8ac00533a0 con 0x7f8ac8129590 2024-04-27T04:34:17.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8abe7fc700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x7f8aac03ceb0 0x7f8aac03f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8aac03f9c0 con 0x7f8aac03ceb0 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac00536a0 con 0x7f8ac8129590 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.928+0000 7f8acd13c700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f8aac037790 0x7f8aac039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.929+0000 7f8ace13e700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x7f8aac03ceb0 0x7f8aac03f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.929+0000 7f8ace13e700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x7f8aac03ceb0 0x7f8aac03f2f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.929+0000 7f8acd13c700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f8aac037790 0x7f8aac039c50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009130 tx=0x7f8ac4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:17.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.930+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8aac03f9c0 con 0x7f8aac03ceb0 2024-04-27T04:34:17.940 INFO:teuthology.orchestra.run.smithi138.stdout:42949672964 2024-04-27T04:34:17.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T04:34:17.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffaf412de20 con 0x7ffae003ceb0 2024-04-27T04:34:17.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffaf0ff9700 1 -- 172.21.15.138:0/3560672417 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffaf412de20 con 0x7ffae003ceb0 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x7ffae003ceb0 msgr2=0x7ffae003f2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x7ffae003ceb0 0x7ffae003f2f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ffae0037790 msgr2=0x7ffae0039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ffae0037790 0x7ffae0039c50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009bc0 tx=0x7ffae8008030 comp rx=0 tx=0).stop 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 msgr2=0x7ffaf41c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.941+0000 7ffafa870700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 0x7ffaf41c6500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffae400c410 tx=0x7ffae400c7d0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 shutdown_connections 2024-04-27T04:34:17.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x7ffae003ceb0 0x7ffae003f2f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaf4097ae0 0x7ffaf41c6500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 --2- 172.21.15.138:0/3560672417 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7ffae0037790 0x7ffae0039c50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 >> 172.21.15.138:0/3560672417 conn(0x7ffaf409e0e0 msgr2=0x7ffaf4095be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 shutdown_connections 2024-04-27T04:34:17.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.942+0000 7ffafa870700 1 -- 172.21.15.138:0/3560672417 wait complete. 2024-04-27T04:34:17.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.947+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ac812a2c0 con 0x7f8aac03ceb0 2024-04-27T04:34:17.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.947+0000 7f8abe7fc700 1 -- 172.21.15.138:0/1602992969 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ac812a2c0 con 0x7f8aac03ceb0 2024-04-27T04:34:17.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x7f8aac03ceb0 msgr2=0x7f8aac03f2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x7f8aac03ceb0 0x7f8aac03f2f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f8aac037790 msgr2=0x7f8aac039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f8aac037790 0x7f8aac039c50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009130 tx=0x7f8ac4008030 comp rx=0 tx=0).stop 2024-04-27T04:34:17.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 msgr2=0x7f8ac81c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 0x7f8ac81c21b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000cb60 tx=0x7f8ac000cf20 comp rx=0 tx=0).stop 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 shutdown_connections 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x7f8aac03ceb0 0x7f8aac03f2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ac8129590 0x7f8ac81c21b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 --2- 172.21.15.138:0/1602992969 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f8aac037790 0x7f8aac039c50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 >> 172.21.15.138:0/1602992969 conn(0x7f8ac809be70 msgr2=0x7f8ac8099020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 shutdown_connections 2024-04-27T04:34:17.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.948+0000 7f8acfba1700 1 -- 172.21.15.138:0/1602992969 wait complete. 2024-04-27T04:34:17.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.954+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f362c09bb40 con 0x7f361803ceb0 2024-04-27T04:34:17.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f3613fff700 1 -- 172.21.15.138:0/318110857 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3624011040 con 0x7f361803ceb0 2024-04-27T04:34:17.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x7f361803ceb0 msgr2=0x7f361803f2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x7f361803ceb0 0x7f361803f2f0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f3618037790 msgr2=0x7f3618039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f3618037790 0x7f3618039c50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f362000be60 tx=0x7f3620014000 comp rx=0 tx=0).stop 2024-04-27T04:34:17.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 msgr2=0x7f362c088fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:17.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 0x7f362c088fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c460 tx=0x7f361c00c820 comp rx=0 tx=0).stop 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 shutdown_connections 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x7f361803ceb0 0x7f361803f2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c12a250 0x7f362c088fc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 --2- 172.21.15.138:0/318110857 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f3618037790 0x7f3618039c50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.955+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 >> 172.21.15.138:0/318110857 conn(0x7f362c09a870 msgr2=0x7f362c08f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.956+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 shutdown_connections 2024-04-27T04:34:17.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:17.956+0000 7f363180f700 1 -- 172.21.15.138:0/318110857 wait complete. 2024-04-27T04:34:17.961 INFO:teuthology.orchestra.run.smithi138.stdout:42949672964 2024-04-27T04:34:17.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T04:34:17.964 INFO:teuthology.orchestra.run.smithi138.stdout:42949672966 2024-04-27T04:34:17.965 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T04:34:17.971 INFO:teuthology.orchestra.run.smithi138.stdout:38654705669 2024-04-27T04:34:17.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T04:34:18.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.072+0000 7f4f42342700 1 Processor -- start 2024-04-27T04:34:18.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.072+0000 7f4f42342700 1 -- start start 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.072+0000 7f4f42342700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4f3c095750 0x7f4f3c095b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.072+0000 7f4f42342700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3c094e60 con 0x7f4f3c097070 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.072+0000 7f4f42342700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3c0977e0 con 0x7f4f3c095750 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3bfff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4f3c095750 0x7f4f3c095b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3b7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4f3c097070 0x7f4f3c096070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43812/0 (socket says 172.21.15.138:43812) 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3b7fe700 1 -- 172.21.15.138:0/2966614547 learned_addr learned my addr 172.21.15.138:0/2966614547 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3bfff700 1 -- 172.21.15.138:0/2966614547 >> v1:172.21.15.138:6789/0 conn(0x7f4f3c097070 legacy=0x7f4f3c096070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3bfff700 1 -- 172.21.15.138:0/2966614547 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c096780 con 0x7f4f3c095750 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3bfff700 1 --2- 172.21.15.138:0/2966614547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c095b30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097c0 tx=0x7f4f2c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d988140f039ce8f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3affd700 1 -- 172.21.15.138:0/2966614547 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2c004160 con 0x7f4f3c095750 2024-04-27T04:34:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3affd700 1 -- 172.21.15.138:0/2966614547 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c0042c0 con 0x7f4f3c095750 2024-04-27T04:34:18.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.073+0000 7f4f3affd700 1 -- 172.21.15.138:0/2966614547 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2c004580 con 0x7f4f3c095750 2024-04-27T04:34:18.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 -- 172.21.15.138:0/2966614547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 msgr2=0x7f4f3c095b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 --2- 172.21.15.138:0/2966614547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c095b30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097c0 tx=0x7f4f2c009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 -- 172.21.15.138:0/2966614547 shutdown_connections 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 --2- 172.21.15.138:0/2966614547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c095b30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 -- 172.21.15.138:0/2966614547 >> 172.21.15.138:0/2966614547 conn(0x7f4f3c09be70 msgr2=0x7f4f3c09c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 -- 172.21.15.138:0/2966614547 shutdown_connections 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.074+0000 7f4f42342700 1 -- 172.21.15.138:0/2966614547 wait complete. 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.076+0000 7f4f42342700 1 Processor -- start 2024-04-27T04:34:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.076+0000 7f4f42342700 1 -- start start 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.076+0000 7f4f42342700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c092e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.076+0000 7f4f42342700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c122ff0 con 0x7f4f3c095750 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f3bfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c092e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f3bfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c092e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47868/0 (socket says 172.21.15.138:47868) 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f3bfff700 1 -- 172.21.15.138:0/746447240 learned_addr learned my addr 172.21.15.138:0/746447240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f3bfff700 1 -- 172.21.15.138:0/746447240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c0094a0 con 0x7f4f3c095750 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f3bfff700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c092e40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c097920 tx=0x7f4f2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2c003df0 con 0x7f4f3c095750 2024-04-27T04:34:18.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c01e070 con 0x7f4f3c095750 2024-04-27T04:34:18.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c072180 con 0x7f4f3c095750 2024-04-27T04:34:18.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2c011780 con 0x7f4f3c095750 2024-04-27T04:34:18.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.077+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c123710 con 0x7f4f3c095750 2024-04-27T04:34:18.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.078+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c124110 con 0x7f4f3c095750 2024-04-27T04:34:18.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.079+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4f2c0119b0 con 0x7f4f3c095750 2024-04-27T04:34:18.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.079+0000 7f4f38ff9700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4f28037730 0x7f4f28039bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.079+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f4f2c04bc10 con 0x7f4f3c095750 2024-04-27T04:34:18.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.079+0000 7f4f3b7fe700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4f28037730 0x7f4f28039bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.080+0000 7f4f3b7fe700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4f28037730 0x7f4f28039bf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4f30001680 tx=0x7f4f30007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.088+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f4f2c0109d0 con 0x7f4f3c095750 2024-04-27T04:34:18.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.099+0000 7f9937754700 1 Processor -- start 2024-04-27T04:34:18.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.099+0000 7f9937754700 1 -- start start 2024-04-27T04:34:18.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.099+0000 7f9937754700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f993012a850 0x7f993012ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f9937754700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300727e0 con 0x7f993012b170 2024-04-27T04:34:18.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f9937754700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993004ebb0 con 0x7f993012a850 2024-04-27T04:34:18.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f99354f0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f993012a850 0x7f993012ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f99354f0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f993012a850 0x7f993012ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47878/0 (socket says 172.21.15.138:47878) 2024-04-27T04:34:18.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f99354f0700 1 -- 172.21.15.138:0/1740117491 learned_addr learned my addr 172.21.15.138:0/1740117491 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f99354f0700 1 -- 172.21.15.138:0/1740117491 >> v1:172.21.15.138:6789/0 conn(0x7f993012b170 legacy=0x7f993012d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f99354f0700 1 -- 172.21.15.138:0/1740117491 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993009bf90 con 0x7f993012a850 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f99354f0700 1 --2- 172.21.15.138:0/1740117491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f993012ac30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f992c009870 tx=0x7f992c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295a90764c1ad1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f9927fff700 1 -- 172.21.15.138:0/1740117491 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f992c004160 con 0x7f993012a850 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f9927fff700 1 -- 172.21.15.138:0/1740117491 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c0042c0 con 0x7f993012a850 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.100+0000 7f9927fff700 1 -- 172.21.15.138:0/1740117491 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f992c004580 con 0x7f993012a850 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 -- 172.21.15.138:0/1740117491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 msgr2=0x7f993012ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 --2- 172.21.15.138:0/1740117491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f993012ac30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f992c009870 tx=0x7f992c009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:18.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 -- 172.21.15.138:0/1740117491 shutdown_connections 2024-04-27T04:34:18.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 --2- 172.21.15.138:0/1740117491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f993012ac30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 -- 172.21.15.138:0/1740117491 >> 172.21.15.138:0/1740117491 conn(0x7f993009a870 msgr2=0x7f993009ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 -- 172.21.15.138:0/1740117491 shutdown_connections 2024-04-27T04:34:18.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.101+0000 7f9937754700 1 -- 172.21.15.138:0/1740117491 wait complete. 2024-04-27T04:34:18.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.102+0000 7f9937754700 1 Processor -- start 2024-04-27T04:34:18.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.102+0000 7f9937754700 1 -- start start 2024-04-27T04:34:18.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f9937754700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f99301c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f9937754700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993008ef40 con 0x7f993012a850 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f99354f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f99301c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f99354f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f99301c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47886/0 (socket says 172.21.15.138:47886) 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f99354f0700 1 -- 172.21.15.138:0/1859277013 learned_addr learned my addr 172.21.15.138:0/1859277013 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f99354f0700 1 -- 172.21.15.138:0/1859277013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c009550 con 0x7f993012a850 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f99354f0700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f99301c2790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f992c009130 tx=0x7f992c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f992c003be0 con 0x7f993012a850 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c01f070 con 0x7f993012a850 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f992c003e70 con 0x7f993012a850 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.103+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9930121400 con 0x7f993012a850 2024-04-27T04:34:18.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.104+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993008e2d0 con 0x7f993012a850 2024-04-27T04:34:18.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.105+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993008f7a0 con 0x7f993012a850 2024-04-27T04:34:18.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.105+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f992c021070 con 0x7f993012a850 2024-04-27T04:34:18.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.105+0000 7f9925ffb700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9914037790 0x7f9914039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.105+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f992c04b2b0 con 0x7f993012a850 2024-04-27T04:34:18.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.105+0000 7f9934cef700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9914037790 0x7f9914039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.106+0000 7f9934cef700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9914037790 0x7f9914039c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9930072fb0 tx=0x7f9920007ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.114+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f992c019780 con 0x7f993012a850 2024-04-27T04:34:18.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.143+0000 7fd0354ac700 1 Processor -- start 2024-04-27T04:34:18.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd0354ac700 1 -- start start 2024-04-27T04:34:18.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd0354ac700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd03012b200 0x7fd03012fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd0354ac700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0300970b0 con 0x7fd03012a710 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd0354ac700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03009db40 con 0x7fd03012b200 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd02effd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd03012a710 0x7fd03012aaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43834/0 (socket says 172.21.15.138:43834) 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd02e7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd03012b200 0x7fd03012fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.144+0000 7fd02effd700 1 -- 172.21.15.138:0/1988068283 learned_addr learned my addr 172.21.15.138:0/1988068283 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02e7fc700 1 -- 172.21.15.138:0/1988068283 >> v1:172.21.15.138:6789/0 conn(0x7fd03012a710 legacy=0x7fd03012aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02dffb700 1 -- 172.21.15.138:0/1988068283 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229172370 0 0) 0x7fd0300970b0 con 0x7fd03012a710 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02e7fc700 1 -- 172.21.15.138:0/1988068283 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030096960 con 0x7fd03012b200 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02e7fc700 1 --2- 172.21.15.138:0/1988068283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012b200 0x7fd03012fa30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd0240097c0 tx=0x7fd024009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a07049bdd64ee82 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02dffb700 1 -- 172.21.15.138:0/1988068283 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd024004160 con 0x7fd03012b200 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02dffb700 1 -- 172.21.15.138:0/1988068283 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0240042c0 con 0x7fd03012b200 2024-04-27T04:34:18.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.145+0000 7fd02dffb700 1 -- 172.21.15.138:0/1988068283 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd024004580 con 0x7fd03012b200 2024-04-27T04:34:18.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 -- 172.21.15.138:0/1988068283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012b200 msgr2=0x7fd03012fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 --2- 172.21.15.138:0/1988068283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012b200 0x7fd03012fa30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd0240097c0 tx=0x7fd024009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 -- 172.21.15.138:0/1988068283 shutdown_connections 2024-04-27T04:34:18.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 --2- 172.21.15.138:0/1988068283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012b200 0x7fd03012fa30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 -- 172.21.15.138:0/1988068283 >> 172.21.15.138:0/1988068283 conn(0x7fd03009d020 msgr2=0x7fd03009d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 -- 172.21.15.138:0/1988068283 shutdown_connections 2024-04-27T04:34:18.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.146+0000 7fd0354ac700 1 -- 172.21.15.138:0/1988068283 wait complete. 2024-04-27T04:34:18.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.147+0000 7fd0354ac700 1 Processor -- start 2024-04-27T04:34:18.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.147+0000 7fd0354ac700 1 -- start start 2024-04-27T04:34:18.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd0354ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 0x7fd03006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd0354ac700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030127e70 con 0x7fd03012a710 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd02effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 0x7fd03006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd02effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 0x7fd03006e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47906/0 (socket says 172.21.15.138:47906) 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd02effd700 1 -- 172.21.15.138:0/4221712938 learned_addr learned my addr 172.21.15.138:0/4221712938 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd02effd700 1 -- 172.21.15.138:0/4221712938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240094a0 con 0x7fd03012a710 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.148+0000 7fd02effd700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 0x7fd03006e6a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd02000c4a0 tx=0x7fd02000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.149+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd020004bb0 con 0x7fd03012a710 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.149+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020004d50 con 0x7fd03012a710 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.149+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd030126f00 con 0x7fd03012a710 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.149+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0200070d0 con 0x7fd03012a710 2024-04-27T04:34:18.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.149+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd030127620 con 0x7fd03012a710 2024-04-27T04:34:18.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.149+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd020007310 con 0x7fd03012a710 2024-04-27T04:34:18.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.150+0000 7fd0177fe700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fd01c037740 0x7fd01c039c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.150+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fd020049d50 con 0x7fd03012a710 2024-04-27T04:34:18.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.150+0000 7fd02e7fc700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fd01c037740 0x7fd01c039c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.150+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd030062510 con 0x7fd03012a710 2024-04-27T04:34:18.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.151+0000 7f861cd31700 1 Processor -- start 2024-04-27T04:34:18.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.151+0000 7f861cd31700 1 -- start start 2024-04-27T04:34:18.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.151+0000 7fd02e7fc700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fd01c037740 0x7fd01c039c00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd024000c00 tx=0x7fd024008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.151+0000 7f861cd31700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8618094380 0x7f8618094760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.151+0000 7f861cd31700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861809cca0 con 0x7f8618094ca0 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.151+0000 7f861cd31700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8618096380 con 0x7f8618094380 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.152+0000 7f861659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8618094380 0x7f8618094760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.152+0000 7f861659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8618094380 0x7f8618094760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47914/0 (socket says 172.21.15.138:47914) 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.152+0000 7f861659c700 1 -- 172.21.15.138:0/3907650078 learned_addr learned my addr 172.21.15.138:0/3907650078 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.152+0000 7f861659c700 1 -- 172.21.15.138:0/3907650078 >> v1:172.21.15.138:6789/0 conn(0x7f8618094ca0 legacy=0x7f861812e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.152+0000 7f861659c700 1 -- 172.21.15.138:0/3907650078 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618099670 con 0x7f8618094380 2024-04-27T04:34:18.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.152+0000 7f861659c700 1 --2- 172.21.15.138:0/3907650078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f8618094760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8608009870 tx=0x7f8608009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de514768db0cd3e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.160+0000 7f861559a700 1 -- 172.21.15.138:0/3907650078 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8608004160 con 0x7f8618094380 2024-04-27T04:34:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.160+0000 7f861559a700 1 -- 172.21.15.138:0/3907650078 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86080042c0 con 0x7f8618094380 2024-04-27T04:34:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.160+0000 7f861559a700 1 -- 172.21.15.138:0/3907650078 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8608004580 con 0x7f8618094380 2024-04-27T04:34:18.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.160+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd0200102c0 con 0x7fd03012a710 2024-04-27T04:34:18.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 -- 172.21.15.138:0/3907650078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 msgr2=0x7f8618094760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 --2- 172.21.15.138:0/3907650078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f8618094760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8608009870 tx=0x7f8608009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 -- 172.21.15.138:0/3907650078 shutdown_connections 2024-04-27T04:34:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 --2- 172.21.15.138:0/3907650078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f8618094760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 -- 172.21.15.138:0/3907650078 >> 172.21.15.138:0/3907650078 conn(0x7f861809c4a0 msgr2=0x7f8618098970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 -- 172.21.15.138:0/3907650078 shutdown_connections 2024-04-27T04:34:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.161+0000 7f861cd31700 1 -- 172.21.15.138:0/3907650078 wait complete. 2024-04-27T04:34:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.162+0000 7f861cd31700 1 Processor -- start 2024-04-27T04:34:18.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.162+0000 7f861cd31700 1 -- start start 2024-04-27T04:34:18.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861cd31700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f861806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861cd31700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618071500 con 0x7f8618094380 2024-04-27T04:34:18.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f861806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f861806fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47916/0 (socket says 172.21.15.138:47916) 2024-04-27T04:34:18.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861659c700 1 -- 172.21.15.138:0/4082319561 learned_addr learned my addr 172.21.15.138:0/4082319561 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:18.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861659c700 1 -- 172.21.15.138:0/4082319561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608009550 con 0x7f8618094380 2024-04-27T04:34:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.163+0000 7f861659c700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f861806fcf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f860800a010 tx=0x7f8608004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.164+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8608003cf0 con 0x7f8618094380 2024-04-27T04:34:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.164+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860801f070 con 0x7f8618094380 2024-04-27T04:34:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.164+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861812e3d0 con 0x7f8618094380 2024-04-27T04:34:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.164+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8608011630 con 0x7f8618094380 2024-04-27T04:34:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.164+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86181215e0 con 0x7f8618094380 2024-04-27T04:34:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.165+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8618067b70 con 0x7f8618094380 2024-04-27T04:34:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.165+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8608021030 con 0x7f8618094380 2024-04-27T04:34:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.165+0000 7f8606ffd700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f85f8037790 0x7f85f8039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.165+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f860804b930 con 0x7f8618094380 2024-04-27T04:34:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.166+0000 7f8615d9b700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f85f8037790 0x7f85f8039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:18.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.166+0000 7f8615d9b700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f85f8037790 0x7f85f8039c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f860c001610 tx=0x7f860c007aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:18.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.170+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f86080117b0 con 0x7f8618094380 2024-04-27T04:34:18.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.302+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4f3c062510 con 0x7f4f3c095750 2024-04-27T04:34:18.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.302+0000 7f4f38ff9700 1 -- 172.21.15.138:0/746447240 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f2c0107f0 con 0x7f4f3c095750 2024-04-27T04:34:18.304 INFO:teuthology.orchestra.run.smithi138.stdout:42949672963 2024-04-27T04:34:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.304+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4f28037730 msgr2=0x7f4f28039bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.304+0000 7f4f42342700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4f28037730 0x7f4f28039bf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4f30001680 tx=0x7f4f30007b50 comp rx=0 tx=0).stop 2024-04-27T04:34:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.304+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 msgr2=0x7f4f3c092e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.304+0000 7f4f42342700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c092e40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c097920 tx=0x7f4f2c004750 comp rx=0 tx=0).stop 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.305+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 shutdown_connections 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.305+0000 7f4f42342700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4f28037730 0x7f4f28039bf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.305+0000 7f4f42342700 1 --2- 172.21.15.138:0/746447240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c095750 0x7f4f3c092e40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.305+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 >> 172.21.15.138:0/746447240 conn(0x7f4f3c09be70 msgr2=0x7f4f3c099020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.305+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 shutdown_connections 2024-04-27T04:34:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.305+0000 7f4f42342700 1 -- 172.21.15.138:0/746447240 wait complete. 2024-04-27T04:34:18.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672964 got 42949672963 for osd.3 2024-04-27T04:34:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.318+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f993008f3d0 con 0x7f993012a850 2024-04-27T04:34:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.318+0000 7f9925ffb700 1 -- 172.21.15.138:0/1859277013 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f992c0193c0 con 0x7f993012a850 2024-04-27T04:34:18.320 INFO:teuthology.orchestra.run.smithi138.stdout:42949672963 2024-04-27T04:34:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9914037790 msgr2=0x7f9914039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9914037790 0x7f9914039c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9930072fb0 tx=0x7f9920007ae0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 msgr2=0x7f99301c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f99301c2790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f992c009130 tx=0x7f992c004750 comp rx=0 tx=0).stop 2024-04-27T04:34:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 shutdown_connections 2024-04-27T04:34:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f9914037790 0x7f9914039c50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 --2- 172.21.15.138:0/1859277013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993012a850 0x7f99301c2790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 >> 172.21.15.138:0/1859277013 conn(0x7f993009a870 msgr2=0x7f9930090340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 shutdown_connections 2024-04-27T04:34:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.320+0000 7f9937754700 1 -- 172.21.15.138:0/1859277013 wait complete. 2024-04-27T04:34:18.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672964 got 42949672963 for osd.1 2024-04-27T04:34:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.363+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd0300708a0 con 0x7fd03012a710 2024-04-27T04:34:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.363+0000 7fd0177fe700 1 -- 172.21.15.138:0/4221712938 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd02001bb30 con 0x7fd03012a710 2024-04-27T04:34:18.365 INFO:teuthology.orchestra.run.smithi138.stdout:38654705667 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fd01c037740 msgr2=0x7fd01c039c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fd01c037740 0x7fd01c039c00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd024000c00 tx=0x7fd024008030 comp rx=0 tx=0).stop 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 msgr2=0x7fd03006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 0x7fd03006e6a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd02000c4a0 tx=0x7fd02000c860 comp rx=0 tx=0).stop 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 shutdown_connections 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fd01c037740 0x7fd01c039c00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 --2- 172.21.15.138:0/4221712938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd03012a710 0x7fd03006e6a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 >> 172.21.15.138:0/4221712938 conn(0x7fd03009d020 msgr2=0x7fd030129dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 shutdown_connections 2024-04-27T04:34:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.365+0000 7fd0354ac700 1 -- 172.21.15.138:0/4221712938 wait complete. 2024-04-27T04:34:18.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.374+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8618067b70 con 0x7f8618094380 2024-04-27T04:34:18.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.374+0000 7f8606ffd700 1 -- 172.21.15.138:0/4082319561 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8608023bc0 con 0x7f8618094380 2024-04-27T04:34:18.376 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2024-04-27T04:34:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f85f8037790 msgr2=0x7f85f8039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f85f8037790 0x7f85f8039c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f860c001610 tx=0x7f860c007aa0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 msgr2=0x7f861806fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f861806fcf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f860800a010 tx=0x7f8608004750 comp rx=0 tx=0).stop 2024-04-27T04:34:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 shutdown_connections 2024-04-27T04:34:18.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f85f8037790 0x7f85f8039c50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 --2- 172.21.15.138:0/4082319561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8618094380 0x7f861806fcf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:18.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 >> 172.21.15.138:0/4082319561 conn(0x7f861809c4a0 msgr2=0x7f8618097700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:18.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 shutdown_connections 2024-04-27T04:34:18.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:18.377+0000 7f861cd31700 1 -- 172.21.15.138:0/4082319561 wait complete. 2024-04-27T04:34:18.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.0 2024-04-27T04:34:18.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672965 for osd.2 2024-04-27T04:34:19.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T04:34:19.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T04:34:19.381 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T04:34:19.391 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T04:34:19.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.461+0000 7f36f9604700 1 Processor -- start 2024-04-27T04:34:19.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.461+0000 7f36f9604700 1 -- start start 2024-04-27T04:34:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f9604700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f36f412a670 0x7f36f412aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f9604700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f40962b0 con 0x7f36f412af90 2024-04-27T04:34:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f9604700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f4092c60 con 0x7f36f412a670 2024-04-27T04:34:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f27fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f36f412af90 0x7f36f4127250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36598/0 (socket says 172.21.15.138:36598) 2024-04-27T04:34:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f2ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f36f412a670 0x7f36f412aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f27fc700 1 -- 172.21.15.138:0/3071114487 learned_addr learned my addr 172.21.15.138:0/3071114487 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f1ffb700 1 -- 172.21.15.138:0/3071114487 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633746222 0 0) 0x7f36f40962b0 con 0x7f36f412af90 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f1ffb700 1 -- 172.21.15.138:0/3071114487 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36dc0035b0 con 0x7f36f412af90 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f2ffd700 1 -- 172.21.15.138:0/3071114487 >> v1:172.21.15.138:6789/0 conn(0x7f36f412af90 legacy=0x7f36f4127250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.462+0000 7f36f2ffd700 1 -- 172.21.15.138:0/3071114487 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f406ab70 con 0x7f36f412a670 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.463+0000 7f36f2ffd700 1 --2- 172.21.15.138:0/3071114487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f412aa50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36e40097c0 tx=0x7f36e4009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a787be0ee38a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.463+0000 7f36f1ffb700 1 -- 172.21.15.138:0/3071114487 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36e4004160 con 0x7f36f412a670 2024-04-27T04:34:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.463+0000 7f36f1ffb700 1 -- 172.21.15.138:0/3071114487 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e40042c0 con 0x7f36f412a670 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.463+0000 7f36f1ffb700 1 -- 172.21.15.138:0/3071114487 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36e4004560 con 0x7f36f412a670 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.463+0000 7f36f9604700 1 -- 172.21.15.138:0/3071114487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 msgr2=0x7f36f412aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.463+0000 7f36f9604700 1 --2- 172.21.15.138:0/3071114487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f412aa50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36e40097c0 tx=0x7f36e4009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.464+0000 7f36f9604700 1 -- 172.21.15.138:0/3071114487 shutdown_connections 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.464+0000 7f36f9604700 1 --2- 172.21.15.138:0/3071114487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f412aa50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.464+0000 7f36f9604700 1 -- 172.21.15.138:0/3071114487 >> 172.21.15.138:0/3071114487 conn(0x7f36f409a450 msgr2=0x7f36f409a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.464+0000 7f36f9604700 1 -- 172.21.15.138:0/3071114487 shutdown_connections 2024-04-27T04:34:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.464+0000 7f36f9604700 1 -- 172.21.15.138:0/3071114487 wait complete. 2024-04-27T04:34:19.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f9604700 1 Processor -- start 2024-04-27T04:34:19.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f9604700 1 -- start start 2024-04-27T04:34:19.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f9604700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f4125600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f9604700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f4092100 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f2ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f4125600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f2ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f4125600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46748/0 (socket says 172.21.15.138:46748) 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.465+0000 7f36f2ffd700 1 -- 172.21.15.138:0/3012117706 learned_addr learned my addr 172.21.15.138:0/3012117706 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36f2ffd700 1 -- 172.21.15.138:0/3012117706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40094a0 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36f2ffd700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f4125600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36e40090b0 tx=0x7f36e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36e4003d00 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e401e070 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f4091dd0 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36e4011630 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.466+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f40914c0 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.467+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f4067b70 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.467+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f36e4011790 con 0x7f36f412a670 2024-04-27T04:34:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.468+0000 7f36db7fe700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f36e0037780 0x7f36e0039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.468+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f36e404b1c0 con 0x7f36f412a670 2024-04-27T04:34:19.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.468+0000 7f36f27fc700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f36e0037780 0x7f36e0039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.468+0000 7f36f27fc700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f36e0037780 0x7f36e0039c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f36e8000f60 tx=0x7f36e80010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.477+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f36e4010740 con 0x7f36f412a670 2024-04-27T04:34:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.509+0000 7f0ed103d700 1 Processor -- start 2024-04-27T04:34:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.509+0000 7f0ed103d700 1 -- start start 2024-04-27T04:34:19.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.510+0000 7f0ed103d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ecc097ae0 0x7f0ecc097ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.510+0000 7f0ed103d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc099a20 con 0x7f0ecc098400 2024-04-27T04:34:19.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.510+0000 7f0ed103d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc098cb0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.510+0000 7f0eca59c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0ecc098400 0x7f0ecc12d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36604/0 (socket says 172.21.15.138:36604) 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.510+0000 7f0eca59c700 1 -- 172.21.15.138:0/1485922626 learned_addr learned my addr 172.21.15.138:0/1485922626 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.510+0000 7f0ecad9d700 1 --2- 172.21.15.138:0/1485922626 >> v2:172.21.15.138:3300/0 conn(0x7f0ecc097ae0 0x7f0ecc097ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ec9d9b700 1 -- 172.21.15.138:0/1485922626 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642978037 0 0) 0x7f0ecc099a20 con 0x7f0ecc098400 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ec9d9b700 1 -- 172.21.15.138:0/1485922626 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb40035b0 con 0x7f0ecc098400 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ecad9d700 1 -- 172.21.15.138:0/1485922626 >> v1:172.21.15.138:6789/0 conn(0x7f0ecc098400 legacy=0x7f0ecc12d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ecad9d700 1 -- 172.21.15.138:0/1485922626 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc099f60 con 0x7f0ecc097ae0 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ecad9d700 1 --2- 172.21.15.138:0/1485922626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc097ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097c0 tx=0x7f0ebc009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d64e6490126a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ec9d9b700 1 -- 172.21.15.138:0/1485922626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc004160 con 0x7f0ecc097ae0 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ec9d9b700 1 -- 172.21.15.138:0/1485922626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc0042c0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.511+0000 7f0ec9d9b700 1 -- 172.21.15.138:0/1485922626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc004580 con 0x7f0ecc097ae0 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 -- 172.21.15.138:0/1485922626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 msgr2=0x7f0ecc097ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 --2- 172.21.15.138:0/1485922626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc097ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097c0 tx=0x7f0ebc009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 -- 172.21.15.138:0/1485922626 shutdown_connections 2024-04-27T04:34:19.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 --2- 172.21.15.138:0/1485922626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc097ec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 -- 172.21.15.138:0/1485922626 >> 172.21.15.138:0/1485922626 conn(0x7f0ecc09e0e0 msgr2=0x7f0ecc09e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 -- 172.21.15.138:0/1485922626 shutdown_connections 2024-04-27T04:34:19.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.512+0000 7f0ed103d700 1 -- 172.21.15.138:0/1485922626 wait complete. 2024-04-27T04:34:19.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.513+0000 7f0ed103d700 1 Processor -- start 2024-04-27T04:34:19.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.513+0000 7f0ed103d700 1 -- start start 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.514+0000 7f0ed103d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc095830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.514+0000 7f0ed103d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1266b0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.514+0000 7f0ecad9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc095830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.514+0000 7f0ecad9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc095830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46770/0 (socket says 172.21.15.138:46770) 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.514+0000 7f0ecad9d700 1 -- 172.21.15.138:0/2141353327 learned_addr learned my addr 172.21.15.138:0/2141353327 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.514+0000 7f0ecad9d700 1 -- 172.21.15.138:0/2141353327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc0094a0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.515+0000 7f0ecad9d700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc095830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009b10 tx=0x7f0ebc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.515+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc003df0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.515+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc01e070 con 0x7f0ecc097ae0 2024-04-27T04:34:19.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.515+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc127140 con 0x7f0ecc097ae0 2024-04-27T04:34:19.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.515+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc0117a0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.515+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc072190 con 0x7f0ecc097ae0 2024-04-27T04:34:19.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.516+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0ebc0119d0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.516+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc127b30 con 0x7f0ecc097ae0 2024-04-27T04:34:19.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.516+0000 7f0eb37fe700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f0eb8037780 0x7f0eb8039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.516+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f0ebc04afb0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.516+0000 7f0eca59c700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f0eb8037780 0x7f0eb8039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.517+0000 7f0eca59c700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f0eb8037780 0x7f0eb8039c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000f60 tx=0x7f0ec00010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.521+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0ebc07e7f0 con 0x7f0ecc097ae0 2024-04-27T04:34:19.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.536+0000 7f50ed0aa700 1 Processor -- start 2024-04-27T04:34:19.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.536+0000 7f799b50b700 1 Processor -- start 2024-04-27T04:34:19.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f799b50b700 1 -- start start 2024-04-27T04:34:19.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f799b50b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79940942b0 0x7f7994131450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.536+0000 7f50ed0aa700 1 -- start start 2024-04-27T04:34:19.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f799b50b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7994094c40 con 0x7f79940727b0 2024-04-27T04:34:19.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f799b50b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7994095a70 con 0x7f79940942b0 2024-04-27T04:34:19.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f50ed0aa700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f50e812a710 0x7f50e812aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f50ed0aa700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80970b0 con 0x7f50e812b030 2024-04-27T04:34:19.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f50ed0aa700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e809db40 con 0x7f50e812a710 2024-04-27T04:34:19.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f50e659c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f50e812b030 0x7f50e8127210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36618/0 (socket says 172.21.15.138:36618) 2024-04-27T04:34:19.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f50e659c700 1 -- 172.21.15.138:0/3128834979 learned_addr learned my addr 172.21.15.138:0/3128834979 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:19.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f7998aa6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79940942b0 0x7f7994131450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f7998aa6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79940942b0 0x7f7994131450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46784/0 (socket says 172.21.15.138:46784) 2024-04-27T04:34:19.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.537+0000 7f7998aa6700 1 -- 172.21.15.138:0/562993148 learned_addr learned my addr 172.21.15.138:0/562993148 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:19.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e6d9d700 1 --2- 172.21.15.138:0/3128834979 >> v2:172.21.15.138:3300/0 conn(0x7f50e812a710 0x7f50e812aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f7998aa6700 1 -- 172.21.15.138:0/562993148 >> v1:172.21.15.138:6789/0 conn(0x7f79940727b0 legacy=0x7f7994093ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T04:34:19.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f7998aa6700 1 -- 172.21.15.138:0/562993148 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799404fb40 con 0x7f79940942b0 2024-04-27T04:34:19.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e5d9b700 1 -- 172.21.15.138:0/3128834979 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344724503 0 0) 0x7f50e80970b0 con 0x7f50e812b030 2024-04-27T04:34:19.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e5d9b700 1 -- 172.21.15.138:0/3128834979 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d00035b0 con 0x7f50e812b030 2024-04-27T04:34:19.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e6d9d700 1 -- 172.21.15.138:0/3128834979 >> v1:172.21.15.138:6789/0 conn(0x7f50e812b030 legacy=0x7f50e8127210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e6d9d700 1 -- 172.21.15.138:0/3128834979 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8096960 con 0x7f50e812a710 2024-04-27T04:34:19.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f7998aa6700 1 --2- 172.21.15.138:0/562993148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940942b0 0x7f7994131450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7984009870 tx=0x7f7984009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78012992c89e00d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e6d9d700 1 --2- 172.21.15.138:0/3128834979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e812aaf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f50d80097c0 tx=0x7f50d8009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428bdfd14ff7f045 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f798bfff700 1 -- 172.21.15.138:0/562993148 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7984004160 con 0x7f79940942b0 2024-04-27T04:34:19.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f798bfff700 1 -- 172.21.15.138:0/562993148 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79840042c0 con 0x7f79940942b0 2024-04-27T04:34:19.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e5d9b700 1 -- 172.21.15.138:0/3128834979 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d8004160 con 0x7f50e812a710 2024-04-27T04:34:19.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e5d9b700 1 -- 172.21.15.138:0/3128834979 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80042c0 con 0x7f50e812a710 2024-04-27T04:34:19.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f798bfff700 1 -- 172.21.15.138:0/562993148 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7984004580 con 0x7f79940942b0 2024-04-27T04:34:19.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.538+0000 7f50e5d9b700 1 -- 172.21.15.138:0/3128834979 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d8004580 con 0x7f50e812a710 2024-04-27T04:34:19.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f50ed0aa700 1 -- 172.21.15.138:0/3128834979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 msgr2=0x7f50e812aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f50ed0aa700 1 --2- 172.21.15.138:0/3128834979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e812aaf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f50d80097c0 tx=0x7f50d8009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f799b50b700 1 -- 172.21.15.138:0/562993148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940942b0 msgr2=0x7f7994131450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f799b50b700 1 --2- 172.21.15.138:0/562993148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940942b0 0x7f7994131450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7984009870 tx=0x7f7984009b80 comp rx=0 tx=0).stop 2024-04-27T04:34:19.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f799b50b700 1 -- 172.21.15.138:0/562993148 shutdown_connections 2024-04-27T04:34:19.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f799b50b700 1 --2- 172.21.15.138:0/562993148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940942b0 0x7f7994131450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f799b50b700 1 -- 172.21.15.138:0/562993148 >> 172.21.15.138:0/562993148 conn(0x7f799409a5d0 msgr2=0x7f799409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f50ed0aa700 1 -- 172.21.15.138:0/3128834979 shutdown_connections 2024-04-27T04:34:19.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f50ed0aa700 1 --2- 172.21.15.138:0/3128834979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e812aaf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.539+0000 7f50ed0aa700 1 -- 172.21.15.138:0/3128834979 >> 172.21.15.138:0/3128834979 conn(0x7f50e809d020 msgr2=0x7f50e809d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.540+0000 7f50ed0aa700 1 -- 172.21.15.138:0/3128834979 shutdown_connections 2024-04-27T04:34:19.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.540+0000 7f50ed0aa700 1 -- 172.21.15.138:0/3128834979 wait complete. 2024-04-27T04:34:19.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.540+0000 7f799b50b700 1 -- 172.21.15.138:0/562993148 shutdown_connections 2024-04-27T04:34:19.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.540+0000 7f799b50b700 1 -- 172.21.15.138:0/562993148 wait complete. 2024-04-27T04:34:19.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f799b50b700 1 Processor -- start 2024-04-27T04:34:19.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f799b50b700 1 -- start start 2024-04-27T04:34:19.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f799b50b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 0x7f79941b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f799b50b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79941b76c0 con 0x7f79940727b0 2024-04-27T04:34:19.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f50ed0aa700 1 Processor -- start 2024-04-27T04:34:19.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f79992a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 0x7f79941b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f50ed0aa700 1 -- start start 2024-04-27T04:34:19.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f79992a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 0x7f79941b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46792/0 (socket says 172.21.15.138:46792) 2024-04-27T04:34:19.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.541+0000 7f79992a7700 1 -- 172.21.15.138:0/3696506692 learned_addr learned my addr 172.21.15.138:0/3696506692 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:19.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f79992a7700 1 -- 172.21.15.138:0/3696506692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984009550 con 0x7f79940727b0 2024-04-27T04:34:19.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f50ed0aa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e81c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f50ed0aa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81c1e70 con 0x7f50e812a710 2024-04-27T04:34:19.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f79992a7700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 0x7f79941b1bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f799000c4d0 tx=0x7f799000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f50e6d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e81c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f50e6d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e81c21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46808/0 (socket says 172.21.15.138:46808) 2024-04-27T04:34:19.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f50e6d9d700 1 -- 172.21.15.138:0/291132334 learned_addr learned my addr 172.21.15.138:0/291132334 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:19.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79900116c0 con 0x7f79940727b0 2024-04-27T04:34:19.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7990011840 con 0x7f79940727b0 2024-04-27T04:34:19.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79941b3da0 con 0x7f79940727b0 2024-04-27T04:34:19.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f50e6d9d700 1 -- 172.21.15.138:0/291132334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80094a0 con 0x7f50e812a710 2024-04-27T04:34:19.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7990011b20 con 0x7f79940727b0 2024-04-27T04:34:19.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.542+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79941b4520 con 0x7f79940727b0 2024-04-27T04:34:19.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.543+0000 7f50e6d9d700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e81c21f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009b10 tx=0x7f50d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.543+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d8003db0 con 0x7f50e812a710 2024-04-27T04:34:19.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.543+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e808adf0 con 0x7f50e812a710 2024-04-27T04:34:19.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.543+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e8126d70 con 0x7f50e812a710 2024-04-27T04:34:19.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.543+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7994067b70 con 0x7f79940727b0 2024-04-27T04:34:19.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.544+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7990011d10 con 0x7f79940727b0 2024-04-27T04:34:19.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.544+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8011630 con 0x7f50e812a710 2024-04-27T04:34:19.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.544+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d80118f0 con 0x7f50e812a710 2024-04-27T04:34:19.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.544+0000 7f7989ffb700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7978037740 0x7f7978039c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.544+0000 7f7998aa6700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7978037740 0x7f7978039c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.545+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e808af80 con 0x7f50e812a710 2024-04-27T04:34:19.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.545+0000 7f7998aa6700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7978037740 0x7f7978039c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f798400a010 tx=0x7f7984008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.545+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f50d8011a50 con 0x7f50e812a710 2024-04-27T04:34:19.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.545+0000 7f50cf7fe700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f50d40377b0 0x7f50d4039c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.545+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f50d804b980 con 0x7f50e812a710 2024-04-27T04:34:19.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.545+0000 7f50e659c700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f50d40377b0 0x7f50d4039c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.546+0000 7f50e659c700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f50d40377b0 0x7f50d4039c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50dc000f60 tx=0x7f50dc0010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.549+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f799000f260 con 0x7f79940727b0 2024-04-27T04:34:19.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.554+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7990012420 con 0x7f79940727b0 2024-04-27T04:34:19.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.563+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f50d8010790 con 0x7f50e812a710 2024-04-27T04:34:19.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.703+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f36f4091c30 con 0x7f36f412a670 2024-04-27T04:34:19.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.704+0000 7f36db7fe700 1 -- 172.21.15.138:0/3012117706 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f36f4091c30 con 0x7f36f412a670 2024-04-27T04:34:19.705 INFO:teuthology.orchestra.run.smithi138.stdout:42949672964 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f36e0037780 msgr2=0x7f36e0039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f36e0037780 0x7f36e0039c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f36e8000f60 tx=0x7f36e80010a0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 msgr2=0x7f36f4125600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f4125600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36e40090b0 tx=0x7f36e4004750 comp rx=0 tx=0).stop 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 shutdown_connections 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f36e0037780 0x7f36e0039c40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 --2- 172.21.15.138:0/3012117706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36f412a670 0x7f36f4125600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 >> 172.21.15.138:0/3012117706 conn(0x7f36f409a450 msgr2=0x7f36f4093aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 shutdown_connections 2024-04-27T04:34:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.707+0000 7f36f9604700 1 -- 172.21.15.138:0/3012117706 wait complete. 2024-04-27T04:34:19.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672964 got 42949672964 for osd.3 2024-04-27T04:34:19.725 DEBUG:teuthology.parallel:result is None 2024-04-27T04:34:19.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.738+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0ecc096680 con 0x7f0ecc097ae0 2024-04-27T04:34:19.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.739+0000 7f0eb37fe700 1 -- 172.21.15.138:0/2141353327 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ecc127b30 con 0x7f0ecc097ae0 2024-04-27T04:34:19.740 INFO:teuthology.orchestra.run.smithi138.stdout:42949672964 2024-04-27T04:34:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f0eb8037780 msgr2=0x7f0eb8039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f0eb8037780 0x7f0eb8039c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000f60 tx=0x7f0ec00010a0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 msgr2=0x7f0ecc095830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc095830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009b10 tx=0x7f0ebc004750 comp rx=0 tx=0).stop 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 shutdown_connections 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f0eb8037780 0x7f0eb8039c40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 --2- 172.21.15.138:0/2141353327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ecc097ae0 0x7f0ecc095830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 >> 172.21.15.138:0/2141353327 conn(0x7f0ecc09e0e0 msgr2=0x7f0ecc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 shutdown_connections 2024-04-27T04:34:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.742+0000 7f0ed103d700 1 -- 172.21.15.138:0/2141353327 wait complete. 2024-04-27T04:34:19.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672964 got 42949672964 for osd.1 2024-04-27T04:34:19.760 DEBUG:teuthology.parallel:result is None 2024-04-27T04:34:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.776+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7994071040 con 0x7f79940727b0 2024-04-27T04:34:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.776+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f50e81278a0 con 0x7f50e812a710 2024-04-27T04:34:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.776+0000 7f7989ffb700 1 -- 172.21.15.138:0/3696506692 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f799001a360 con 0x7f79940727b0 2024-04-27T04:34:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:42949672966 2024-04-27T04:34:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.777+0000 7f50cf7fe700 1 -- 172.21.15.138:0/291132334 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50d8010380 con 0x7f50e812a710 2024-04-27T04:34:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:38654705669 2024-04-27T04:34:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7978037740 msgr2=0x7f7978039c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7978037740 0x7f7978039c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f798400a010 tx=0x7f7984008030 comp rx=0 tx=0).stop 2024-04-27T04:34:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 msgr2=0x7f79941b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 0x7f79941b1bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f799000c4d0 tx=0x7f799000c890 comp rx=0 tx=0).stop 2024-04-27T04:34:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f50d40377b0 msgr2=0x7f50d4039c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 shutdown_connections 2024-04-27T04:34:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7978037740 0x7f7978039c00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f50d40377b0 0x7f50d4039c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50dc000f60 tx=0x7f50dc0010a0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 msgr2=0x7f50e81c21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e81c21f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009b10 tx=0x7f50d80047c0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 --2- 172.21.15.138:0/3696506692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79940727b0 0x7f79941b1bd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 >> 172.21.15.138:0/3696506692 conn(0x7f799409a5d0 msgr2=0x7f79941b9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 shutdown_connections 2024-04-27T04:34:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f50d40377b0 0x7f50d4039c70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 --2- 172.21.15.138:0/291132334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e812a710 0x7f50e81c21f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 >> 172.21.15.138:0/291132334 conn(0x7f50e809d020 msgr2=0x7f50e806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 shutdown_connections 2024-04-27T04:34:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f799b50b700 1 -- 172.21.15.138:0/3696506692 wait complete. 2024-04-27T04:34:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 shutdown_connections 2024-04-27T04:34:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.779+0000 7f50ed0aa700 1 -- 172.21.15.138:0/291132334 wait complete. 2024-04-27T04:34:19.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.2 2024-04-27T04:34:19.794 DEBUG:teuthology.parallel:result is None 2024-04-27T04:34:19.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-04-27T04:34:19.794 DEBUG:teuthology.parallel:result is None 2024-04-27T04:34:19.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T04:34:19.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T04:34:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fd19ad700 1 Processor -- start 2024-04-27T04:34:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fd19ad700 1 -- start start 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fd19ad700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4fcc130c60 0x7f4fcc133050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fd19ad700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc099a20 con 0x7f4fcc097ae0 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fd19ad700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc098cb0 con 0x7f4fcc130c60 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fca7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4fcc130c60 0x7f4fcc133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fcaffd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4fcc097ae0 0x7f4fcc097ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36642/0 (socket says 172.21.15.138:36642) 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.951+0000 7f4fcaffd700 1 -- 172.21.15.138:0/1097498241 learned_addr learned my addr 172.21.15.138:0/1097498241 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fca7fc700 1 -- 172.21.15.138:0/1097498241 >> v1:172.21.15.138:6789/0 conn(0x7f4fcc097ae0 legacy=0x7f4fcc097ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fca7fc700 1 -- 172.21.15.138:0/1097498241 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc099f60 con 0x7f4fcc130c60 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fc9ffb700 1 -- 172.21.15.138:0/1097498241 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020079470 0 0) 0x7f4fcc099a20 con 0x7f4fcc097ae0 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fca7fc700 1 --2- 172.21.15.138:0/1097498241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc130c60 0x7f4fcc133050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00097c0 tx=0x7f4fc0009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06cfa6883a741fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fc9ffb700 1 -- 172.21.15.138:0/1097498241 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fc0004160 con 0x7f4fcc130c60 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fc9ffb700 1 -- 172.21.15.138:0/1097498241 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc00042c0 con 0x7f4fcc130c60 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fc9ffb700 1 -- 172.21.15.138:0/1097498241 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fc0004580 con 0x7f4fcc130c60 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 -- 172.21.15.138:0/1097498241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc130c60 msgr2=0x7f4fcc133050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 --2- 172.21.15.138:0/1097498241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc130c60 0x7f4fcc133050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00097c0 tx=0x7f4fc0009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 -- 172.21.15.138:0/1097498241 shutdown_connections 2024-04-27T04:34:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 --2- 172.21.15.138:0/1097498241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc130c60 0x7f4fcc133050 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 -- 172.21.15.138:0/1097498241 >> 172.21.15.138:0/1097498241 conn(0x7f4fcc09e0e0 msgr2=0x7f4fcc09e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 -- 172.21.15.138:0/1097498241 shutdown_connections 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.952+0000 7f4fd19ad700 1 -- 172.21.15.138:0/1097498241 wait complete. 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.953+0000 7f4fd19ad700 1 Processor -- start 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.953+0000 7f4fd19ad700 1 -- start start 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fd19ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 0x7f4fcc06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fd19ad700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc071330 con 0x7f4fcc097ae0 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fcaffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 0x7f4fcc06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fcaffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 0x7f4fcc06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46822/0 (socket says 172.21.15.138:46822) 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fcaffd700 1 -- 172.21.15.138:0/475153554 learned_addr learned my addr 172.21.15.138:0/475153554 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fcaffd700 1 -- 172.21.15.138:0/475153554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00094a0 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fcaffd700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 0x7f4fcc06ebb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c4a0 tx=0x7f4fbc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc004bb0 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc004d10 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc0706f0 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc0070d0 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.954+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc12b0a0 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.955+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4fbc0072e0 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.955+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc072210 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.955+0000 7f4fb3fff700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4fb8037790 0x7f4fb8039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.955+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f4fbc049c80 con 0x7f4fcc097ae0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.956+0000 7f4fca7fc700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4fb8037790 0x7f4fb8039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.956+0000 7f4fca7fc700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4fb8037790 0x7f4fb8039c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009b10 tx=0x7f4fc0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:19.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:19.960+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f4fbc04f020 con 0x7f4fcc097ae0 2024-04-27T04:34:20.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.148+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 --> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fcc0985d0 con 0x7f4fb8037790 2024-04-27T04:34:20.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.150+0000 7f4fb3fff700 1 -- 172.21.15.138:0/475153554 <== mgr.4100 v2:172.21.15.191:6824/41511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34631 (secure 0 0 0) 0x7f4fc001e070 con 0x7f4fb8037790 2024-04-27T04:34:20.152 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4fb8037790 msgr2=0x7f4fb8039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4fb8037790 0x7f4fb8039c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009b10 tx=0x7f4fc0008030 comp rx=0 tx=0).stop 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 msgr2=0x7f4fcc06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 0x7f4fcc06ebb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c4a0 tx=0x7f4fbc00c860 comp rx=0 tx=0).stop 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 shutdown_connections 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f4fb8037790 0x7f4fb8039c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 --2- 172.21.15.138:0/475153554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fcc097ae0 0x7f4fcc06ebb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.151+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 >> 172.21.15.138:0/475153554 conn(0x7f4fcc09e0e0 msgr2=0x7f4fcc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.152+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 shutdown_connections 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.152+0000 7f4fd19ad700 1 -- 172.21.15.138:0/475153554 wait complete. 2024-04-27T04:34:20.154 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-04-27T04:34:20.165 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-27T04:34:19.092298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":109764,"kb_used_data":1672,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377596,"statfs":{"total":386547056640,"available":386434658304,"internally_reserved":0,"allocated":1712128,"data_stored":1338468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6373,"internal_metadata":110618395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000788"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:17.178645+0000","last_change":"2024-04-27T04:34:17.178120+0000","last_active":"2024-04-27T04:34:17.178645+0000","last_peered":"2024-04-27T04:34:17.178645+0000","last_clean":"2024-04-27T04:34:17.178645+0000","last_became_active":"2024-04-27T04:34:17.177794+0000","last_became_peered":"2024-04-27T04:34:17.177794+0000","last_unstale":"2024-04-27T04:34:17.178645+0000","last_undegraded":"2024-04-27T04:34:17.178645+0000","last_fullsized":"2024-04-27T04:34:17.178645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:24.870492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:17.093729+0000","last_change":"2024-04-27T04:34:16.689225+0000","last_active":"2024-04-27T04:34:17.093729+0000","last_peered":"2024-04-27T04:34:17.093729+0000","last_clean":"2024-04-27T04:34:17.093729+0000","last_became_active":"2024-04-27T04:34:16.688730+0000","last_became_peered":"2024-04-27T04:34:16.688730+0000","last_unstale":"2024-04-27T04:34:17.093729+0000","last_undegraded":"2024-04-27T04:34:17.093729+0000","last_fullsized":"2024-04-27T04:34:17.093729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:08.707366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.689738+0000","last_change":"2024-04-27T04:34:16.688297+0000","last_active":"2024-04-27T04:34:16.689738+0000","last_peered":"2024-04-27T04:34:16.689738+0000","last_clean":"2024-04-27T04:34:16.689738+0000","last_became_active":"2024-04-27T04:34:16.687940+0000","last_became_peered":"2024-04-27T04:34:16.687940+0000","last_unstale":"2024-04-27T04:34:16.689738+0000","last_undegraded":"2024-04-27T04:34:16.689738+0000","last_fullsized":"2024-04-27T04:34:16.689738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:18.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.689774+0000","last_change":"2024-04-27T04:34:16.688520+0000","last_active":"2024-04-27T04:34:16.689774+0000","last_peered":"2024-04-27T04:34:16.689774+0000","last_clean":"2024-04-27T04:34:16.689774+0000","last_became_active":"2024-04-27T04:34:16.688245+0000","last_became_peered":"2024-04-27T04:34:16.688245+0000","last_unstale":"2024-04-27T04:34:16.689774+0000","last_undegraded":"2024-04-27T04:34:16.689774+0000","last_fullsized":"2024-04-27T04:34:16.689774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:12:21.414106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-27T04:34:15.685590+0000","last_change":"2024-04-27T04:34:10.451955+0000","last_active":"2024-04-27T04:34:15.685590+0000","last_peered":"2024-04-27T04:34:15.685590+0000","last_clean":"2024-04-27T04:34:15.685590+0000","last_became_active":"2024-04-27T04:34:10.451048+0000","last_became_peered":"2024-04-27T04:34:10.451048+0000","last_unstale":"2024-04-27T04:34:15.685590+0000","last_undegraded":"2024-04-27T04:34:15.685590+0000","last_fullsized":"2024-04-27T04:34:15.685590+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:09.435938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:09.435938+0000","last_clean_scrub_stamp":"2024-04-27T04:34:09.435938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:59:30.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.688830+0000","last_change":"2024-04-27T04:34:16.687524+0000","last_active":"2024-04-27T04:34:16.688830+0000","last_peered":"2024-04-27T04:34:16.688830+0000","last_clean":"2024-04-27T04:34:16.688830+0000","last_became_active":"2024-04-27T04:34:16.687211+0000","last_became_peered":"2024-04-27T04:34:16.687211+0000","last_unstale":"2024-04-27T04:34:16.688830+0000","last_undegraded":"2024-04-27T04:34:16.688830+0000","last_fullsized":"2024-04-27T04:34:16.688830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:32:32.286334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.690262+0000","last_change":"2024-04-27T04:34:16.688139+0000","last_active":"2024-04-27T04:34:16.690262+0000","last_peered":"2024-04-27T04:34:16.690262+0000","last_clean":"2024-04-27T04:34:16.690262+0000","last_became_active":"2024-04-27T04:34:16.687773+0000","last_became_peered":"2024-04-27T04:34:16.687773+0000","last_unstale":"2024-04-27T04:34:16.690262+0000","last_undegraded":"2024-04-27T04:34:16.690262+0000","last_fullsized":"2024-04-27T04:34:16.690262+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:31.602425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.689738+0000","last_change":"2024-04-27T04:34:16.688202+0000","last_active":"2024-04-27T04:34:16.689738+0000","last_peered":"2024-04-27T04:34:16.689738+0000","last_clean":"2024-04-27T04:34:16.689738+0000","last_became_active":"2024-04-27T04:34:16.687891+0000","last_became_peered":"2024-04-27T04:34:16.687891+0000","last_unstale":"2024-04-27T04:34:16.689738+0000","last_undegraded":"2024-04-27T04:34:16.689738+0000","last_fullsized":"2024-04-27T04:34:16.689738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:36:55.370738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:17.093712+0000","last_change":"2024-04-27T04:34:16.689366+0000","last_active":"2024-04-27T04:34:17.093712+0000","last_peered":"2024-04-27T04:34:17.093712+0000","last_clean":"2024-04-27T04:34:17.093712+0000","last_became_active":"2024-04-27T04:34:16.689082+0000","last_became_peered":"2024-04-27T04:34:16.689082+0000","last_unstale":"2024-04-27T04:34:17.093712+0000","last_undegraded":"2024-04-27T04:34:17.093712+0000","last_fullsized":"2024-04-27T04:34:17.093712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:49.579210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":39433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":629801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":39433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]}]},{"osd":1,"up_from":10,"seq":42949672964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":629801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T04:34:20.166 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T04:34:20.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.283+0000 7f6626ee0700 1 Processor -- start 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.283+0000 7f6626ee0700 1 -- start start 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.283+0000 7f6626ee0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6620129590 0x7f6620129970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.283+0000 7f6626ee0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6620097a00 con 0x7f6620129eb0 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.283+0000 7f6626ee0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6620067300 con 0x7f6620129590 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f6624c7c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6620129590 0x7f6620129970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f661ffff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6620129eb0 0x7f662012a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36652/0 (socket says 172.21.15.138:36652) 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f661ffff700 1 -- 172.21.15.138:0/2038905713 learned_addr learned my addr 172.21.15.138:0/2038905713 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f6624c7c700 1 -- 172.21.15.138:0/2038905713 >> v1:172.21.15.138:6789/0 conn(0x7f6620129eb0 legacy=0x7f662012a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f6624c7c700 1 -- 172.21.15.138:0/2038905713 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620097f40 con 0x7f6620129590 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f661f7fe700 1 -- 172.21.15.138:0/2038905713 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183117253 0 0) 0x7f6620097a00 con 0x7f6620129eb0 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f6624c7c700 1 --2- 172.21.15.138:0/2038905713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620129970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66140097c0 tx=0x7f6614009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fb1da206c2360a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f661f7fe700 1 -- 172.21.15.138:0/2038905713 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6614004160 con 0x7f6620129590 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f661f7fe700 1 -- 172.21.15.138:0/2038905713 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66140042c0 con 0x7f6620129590 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.284+0000 7f661f7fe700 1 -- 172.21.15.138:0/2038905713 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6614004580 con 0x7f6620129590 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 -- 172.21.15.138:0/2038905713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 msgr2=0x7f6620129970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 --2- 172.21.15.138:0/2038905713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620129970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66140097c0 tx=0x7f6614009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 -- 172.21.15.138:0/2038905713 shutdown_connections 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 --2- 172.21.15.138:0/2038905713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620129970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 -- 172.21.15.138:0/2038905713 >> 172.21.15.138:0/2038905713 conn(0x7f662009be70 msgr2=0x7f662009c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 -- 172.21.15.138:0/2038905713 shutdown_connections 2024-04-27T04:34:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.285+0000 7f6626ee0700 1 -- 172.21.15.138:0/2038905713 wait complete. 2024-04-27T04:34:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6626ee0700 1 Processor -- start 2024-04-27T04:34:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6626ee0700 1 -- start start 2024-04-27T04:34:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6626ee0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620127780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6626ee0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662006f590 con 0x7f6620129590 2024-04-27T04:34:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6624c7c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620127780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6624c7c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620127780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46850/0 (socket says 172.21.15.138:46850) 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6624c7c700 1 -- 172.21.15.138:0/4048007548 learned_addr learned my addr 172.21.15.138:0/4048007548 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.286+0000 7f6624c7c700 1 -- 172.21.15.138:0/4048007548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140094a0 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f6624c7c700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620127780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6614009b10 tx=0x7f6614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6614003df0 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661401e070 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6614011760 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6620127160 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6620126390 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.287+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6614011990 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.288+0000 7f661d7fa700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f6604037780 0x7f6604039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.288+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6620126f80 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.288+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f661404b370 con 0x7f6620129590 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.288+0000 7f661ffff700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f6604037780 0x7f6604039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.288+0000 7f661ffff700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f6604037780 0x7f6604039c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f660c000c00 tx=0x7f660c007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.292+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f6614010970 con 0x7f6620129590 2024-04-27T04:34:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.480+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 --> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f662012a2d0 con 0x7f6604037780 2024-04-27T04:34:20.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.482+0000 7f661d7fa700 1 -- 172.21.15.138:0/4048007548 <== mgr.4100 v2:172.21.15.191:6824/41511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34631 (secure 0 0 0) 0x7f660c001370 con 0x7f6604037780 2024-04-27T04:34:20.484 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f6604037780 msgr2=0x7f6604039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f6604037780 0x7f6604039c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f660c000c00 tx=0x7f660c007a00 comp rx=0 tx=0).stop 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 msgr2=0x7f6620127780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620127780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6614009b10 tx=0x7f6614004750 comp rx=0 tx=0).stop 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 shutdown_connections 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f6604037780 0x7f6604039c40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 --2- 172.21.15.138:0/4048007548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6620129590 0x7f6620127780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 >> 172.21.15.138:0/4048007548 conn(0x7f662009be70 msgr2=0x7f6620099480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 shutdown_connections 2024-04-27T04:34:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.484+0000 7f6626ee0700 1 -- 172.21.15.138:0/4048007548 wait complete. 2024-04-27T04:34:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-04-27T04:34:20.497 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-27T04:34:19.092298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":109764,"kb_used_data":1672,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377596,"statfs":{"total":386547056640,"available":386434658304,"internally_reserved":0,"allocated":1712128,"data_stored":1338468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6373,"internal_metadata":110618395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000788"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:17.178645+0000","last_change":"2024-04-27T04:34:17.178120+0000","last_active":"2024-04-27T04:34:17.178645+0000","last_peered":"2024-04-27T04:34:17.178645+0000","last_clean":"2024-04-27T04:34:17.178645+0000","last_became_active":"2024-04-27T04:34:17.177794+0000","last_became_peered":"2024-04-27T04:34:17.177794+0000","last_unstale":"2024-04-27T04:34:17.178645+0000","last_undegraded":"2024-04-27T04:34:17.178645+0000","last_fullsized":"2024-04-27T04:34:17.178645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:24.870492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:17.093729+0000","last_change":"2024-04-27T04:34:16.689225+0000","last_active":"2024-04-27T04:34:17.093729+0000","last_peered":"2024-04-27T04:34:17.093729+0000","last_clean":"2024-04-27T04:34:17.093729+0000","last_became_active":"2024-04-27T04:34:16.688730+0000","last_became_peered":"2024-04-27T04:34:16.688730+0000","last_unstale":"2024-04-27T04:34:17.093729+0000","last_undegraded":"2024-04-27T04:34:17.093729+0000","last_fullsized":"2024-04-27T04:34:17.093729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:08.707366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.689738+0000","last_change":"2024-04-27T04:34:16.688297+0000","last_active":"2024-04-27T04:34:16.689738+0000","last_peered":"2024-04-27T04:34:16.689738+0000","last_clean":"2024-04-27T04:34:16.689738+0000","last_became_active":"2024-04-27T04:34:16.687940+0000","last_became_peered":"2024-04-27T04:34:16.687940+0000","last_unstale":"2024-04-27T04:34:16.689738+0000","last_undegraded":"2024-04-27T04:34:16.689738+0000","last_fullsized":"2024-04-27T04:34:16.689738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:18.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.689774+0000","last_change":"2024-04-27T04:34:16.688520+0000","last_active":"2024-04-27T04:34:16.689774+0000","last_peered":"2024-04-27T04:34:16.689774+0000","last_clean":"2024-04-27T04:34:16.689774+0000","last_became_active":"2024-04-27T04:34:16.688245+0000","last_became_peered":"2024-04-27T04:34:16.688245+0000","last_unstale":"2024-04-27T04:34:16.689774+0000","last_undegraded":"2024-04-27T04:34:16.689774+0000","last_fullsized":"2024-04-27T04:34:16.689774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:12:21.414106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-27T04:34:15.685590+0000","last_change":"2024-04-27T04:34:10.451955+0000","last_active":"2024-04-27T04:34:15.685590+0000","last_peered":"2024-04-27T04:34:15.685590+0000","last_clean":"2024-04-27T04:34:15.685590+0000","last_became_active":"2024-04-27T04:34:10.451048+0000","last_became_peered":"2024-04-27T04:34:10.451048+0000","last_unstale":"2024-04-27T04:34:15.685590+0000","last_undegraded":"2024-04-27T04:34:15.685590+0000","last_fullsized":"2024-04-27T04:34:15.685590+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:09.435938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:09.435938+0000","last_clean_scrub_stamp":"2024-04-27T04:34:09.435938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:59:30.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.688830+0000","last_change":"2024-04-27T04:34:16.687524+0000","last_active":"2024-04-27T04:34:16.688830+0000","last_peered":"2024-04-27T04:34:16.688830+0000","last_clean":"2024-04-27T04:34:16.688830+0000","last_became_active":"2024-04-27T04:34:16.687211+0000","last_became_peered":"2024-04-27T04:34:16.687211+0000","last_unstale":"2024-04-27T04:34:16.688830+0000","last_undegraded":"2024-04-27T04:34:16.688830+0000","last_fullsized":"2024-04-27T04:34:16.688830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:32:32.286334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.690262+0000","last_change":"2024-04-27T04:34:16.688139+0000","last_active":"2024-04-27T04:34:16.690262+0000","last_peered":"2024-04-27T04:34:16.690262+0000","last_clean":"2024-04-27T04:34:16.690262+0000","last_became_active":"2024-04-27T04:34:16.687773+0000","last_became_peered":"2024-04-27T04:34:16.687773+0000","last_unstale":"2024-04-27T04:34:16.690262+0000","last_undegraded":"2024-04-27T04:34:16.690262+0000","last_fullsized":"2024-04-27T04:34:16.690262+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:31.602425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:16.689738+0000","last_change":"2024-04-27T04:34:16.688202+0000","last_active":"2024-04-27T04:34:16.689738+0000","last_peered":"2024-04-27T04:34:16.689738+0000","last_clean":"2024-04-27T04:34:16.689738+0000","last_became_active":"2024-04-27T04:34:16.687891+0000","last_became_peered":"2024-04-27T04:34:16.687891+0000","last_unstale":"2024-04-27T04:34:16.689738+0000","last_undegraded":"2024-04-27T04:34:16.689738+0000","last_fullsized":"2024-04-27T04:34:16.689738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:36:55.370738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T04:34:17.093712+0000","last_change":"2024-04-27T04:34:16.689366+0000","last_active":"2024-04-27T04:34:17.093712+0000","last_peered":"2024-04-27T04:34:17.093712+0000","last_clean":"2024-04-27T04:34:17.093712+0000","last_became_active":"2024-04-27T04:34:16.689082+0000","last_became_peered":"2024-04-27T04:34:16.689082+0000","last_unstale":"2024-04-27T04:34:17.093712+0000","last_undegraded":"2024-04-27T04:34:17.093712+0000","last_fullsized":"2024-04-27T04:34:17.093712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T04:34:15.677647+0000","last_clean_scrub_stamp":"2024-04-27T04:34:15.677647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:49.579210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":39433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":629801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":39433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]}]},{"osd":1,"up_from":10,"seq":42949672964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":629801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T04:34:20.498 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T04:34:20.498 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-27T04:34:20.498 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-27T04:34:20.498 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-27T04:34:20.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db782f700 1 Processor -- start 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db782f700 1 -- start start 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db782f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2db0129590 0x7f2db0129970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db782f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db0097a00 con 0x7f2db0129eb0 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db782f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db0067300 con 0x7f2db0129590 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db55cb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2db0129590 0x7f2db0129970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db4dca700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2db0129eb0 0x7f2db012a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36668/0 (socket says 172.21.15.138:36668) 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.615+0000 7f2db4dca700 1 -- 172.21.15.138:0/3874535164 learned_addr learned my addr 172.21.15.138:0/3874535164 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db55cb700 1 -- 172.21.15.138:0/3874535164 >> v1:172.21.15.138:6789/0 conn(0x7f2db0129eb0 legacy=0x7f2db012a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2da7fff700 1 -- 172.21.15.138:0/3874535164 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579037546 0 0) 0x7f2db0097a00 con 0x7f2db0129eb0 2024-04-27T04:34:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db55cb700 1 -- 172.21.15.138:0/3874535164 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0097f40 con 0x7f2db0129590 2024-04-27T04:34:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db55cb700 1 --2- 172.21.15.138:0/3874535164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0129970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0097c0 tx=0x7f2dac009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6939b901745cf31d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2da7fff700 1 -- 172.21.15.138:0/3874535164 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dac004160 con 0x7f2db0129590 2024-04-27T04:34:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2da7fff700 1 -- 172.21.15.138:0/3874535164 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac0042c0 con 0x7f2db0129590 2024-04-27T04:34:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2da7fff700 1 -- 172.21.15.138:0/3874535164 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dac004580 con 0x7f2db0129590 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db782f700 1 -- 172.21.15.138:0/3874535164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 msgr2=0x7f2db0129970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db782f700 1 --2- 172.21.15.138:0/3874535164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0129970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0097c0 tx=0x7f2dac009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db782f700 1 -- 172.21.15.138:0/3874535164 shutdown_connections 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db782f700 1 --2- 172.21.15.138:0/3874535164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0129970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.616+0000 7f2db782f700 1 -- 172.21.15.138:0/3874535164 >> 172.21.15.138:0/3874535164 conn(0x7f2db009be70 msgr2=0x7f2db009c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.617+0000 7f2db782f700 1 -- 172.21.15.138:0/3874535164 shutdown_connections 2024-04-27T04:34:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.617+0000 7f2db782f700 1 -- 172.21.15.138:0/3874535164 wait complete. 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db782f700 1 Processor -- start 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db782f700 1 -- start start 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db782f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0126180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db782f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00890e0 con 0x7f2db0129590 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db55cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0126180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db55cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0126180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46862/0 (socket says 172.21.15.138:46862) 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.618+0000 7f2db55cb700 1 -- 172.21.15.138:0/2109439473 learned_addr learned my addr 172.21.15.138:0/2109439473 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2db55cb700 1 -- 172.21.15.138:0/2109439473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac0094a0 con 0x7f2db0129590 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2db55cb700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0126180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dac006ec0 tx=0x7f2dac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dac011630 con 0x7f2db0129590 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac01e070 con 0x7f2db0129590 2024-04-27T04:34:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db0127350 con 0x7f2db0129590 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dac0117b0 con 0x7f2db0129590 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.619+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db0128670 con 0x7f2db0129590 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.620+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2dac011960 con 0x7f2db0129590 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.620+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db0089650 con 0x7f2db0129590 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.620+0000 7f2da5ffb700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d94037780 0x7f2d94039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.620+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f2dac04b2e0 con 0x7f2db0129590 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.620+0000 7f2db4dca700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d94037780 0x7f2d94039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.621+0000 7f2db4dca700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d94037780 0x7f2d94039c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2da0000c00 tx=0x7f2da0007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:20.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.625+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2dac018350 con 0x7f2db0129590 2024-04-27T04:34:20.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.861+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2db0062510 con 0x7f2db0129590 2024-04-27T04:34:20.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.861+0000 7f2da5ffb700 1 -- 172.21.15.138:0/2109439473 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2dac010b70 con 0x7f2db0129590 2024-04-27T04:34:20.864 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T04:34:20.864 INFO:teuthology.orchestra.run.smithi138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-27T04:34:20.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.863+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d94037780 msgr2=0x7f2d94039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.863+0000 7f2db782f700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d94037780 0x7f2d94039c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2da0000c00 tx=0x7f2da0007a00 comp rx=0 tx=0).stop 2024-04-27T04:34:20.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.863+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 msgr2=0x7f2db0126180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.863+0000 7f2db782f700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0126180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dac006ec0 tx=0x7f2dac004750 comp rx=0 tx=0).stop 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.864+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 shutdown_connections 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.864+0000 7f2db782f700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d94037780 0x7f2d94039c40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.864+0000 7f2db782f700 1 --2- 172.21.15.138:0/2109439473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0129590 0x7f2db0126180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.864+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 >> 172.21.15.138:0/2109439473 conn(0x7f2db009be70 msgr2=0x7f2db0099510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.864+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 shutdown_connections 2024-04-27T04:34:20.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:34:20.864+0000 7f2db782f700 1 -- 172.21.15.138:0/2109439473 wait complete. 2024-04-27T04:34:20.877 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-27T04:34:20.877 INFO:teuthology.run_tasks:Running task rgw... 2024-04-27T04:34:20.892 DEBUG:tasks.rgw:config is {'client.0': None} 2024-04-27T04:34:20.892 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-04-27T04:34:20.892 INFO:tasks.rgw:Creating data pools 2024-04-27T04:34:20.892 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-04-27T04:34:20.893 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.026+0000 7fdf600d4700 1 Processor -- start 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf600d4700 1 -- start start 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf600d4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf581150e0 0x7fdf581154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf600d4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf58115b50 con 0x7fdf5810ed90 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf600d4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5810f950 con 0x7fdf581150e0 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf5de70700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf581150e0 0x7fdf581154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:21.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf5d66f700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdf5810ed90 0x7fdf5810f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:48330/0 (socket says 172.21.15.191:48330) 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf5d66f700 1 -- 172.21.15.191:0/4118769409 learned_addr learned my addr 172.21.15.191:0/4118769409 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.027+0000 7fdf5ce6e700 1 -- 172.21.15.191:0/4118769409 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226148597 0 0) 0x7fdf58115b50 con 0x7fdf5810ed90 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5ce6e700 1 -- 172.21.15.191:0/4118769409 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf440035b0 con 0x7fdf5810ed90 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5de70700 1 -- 172.21.15.191:0/4118769409 >> v1:172.21.15.138:6789/0 conn(0x7fdf5810ed90 legacy=0x7fdf5810f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5de70700 1 -- 172.21.15.191:0/4118769409 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5810fac0 con 0x7fdf581150e0 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5de70700 1 --2- 172.21.15.191:0/4118769409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf581150e0 0x7fdf581154c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf480097c0 tx=0x7fdf48009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fe662e33d5694f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5ce6e700 1 -- 172.21.15.191:0/4118769409 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf48004160 con 0x7fdf581150e0 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5ce6e700 1 -- 172.21.15.191:0/4118769409 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf480042c0 con 0x7fdf581150e0 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf5ce6e700 1 -- 172.21.15.191:0/4118769409 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf48004560 con 0x7fdf581150e0 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf600d4700 1 -- 172.21.15.191:0/4118769409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf581150e0 msgr2=0x7fdf581154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf600d4700 1 --2- 172.21.15.191:0/4118769409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf581150e0 0x7fdf581154c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf480097c0 tx=0x7fdf48009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf600d4700 1 -- 172.21.15.191:0/4118769409 shutdown_connections 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf600d4700 1 --2- 172.21.15.191:0/4118769409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf581150e0 0x7fdf581154c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf600d4700 1 -- 172.21.15.191:0/4118769409 >> 172.21.15.191:0/4118769409 conn(0x7fdf58070e00 msgr2=0x7fdf58071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:21.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.028+0000 7fdf600d4700 1 -- 172.21.15.191:0/4118769409 shutdown_connections 2024-04-27T04:34:21.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf600d4700 1 -- 172.21.15.191:0/4118769409 wait complete. 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf600d4700 1 Processor -- start 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf600d4700 1 -- start start 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf600d4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 0x7fdf581a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf600d4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581a9780 con 0x7fdf5810ed90 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf5de70700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 0x7fdf581a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf5de70700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 0x7fdf581a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49402/0 (socket says 172.21.15.191:49402) 2024-04-27T04:34:21.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.029+0000 7fdf5de70700 1 -- 172.21.15.191:0/884246 learned_addr learned my addr 172.21.15.191:0/884246 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf5de70700 1 -- 172.21.15.191:0/884246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480094a0 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf5de70700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 0x7fdf581a91b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009fd0 tx=0x7fdf48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf48003d00 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4801e070 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581a9920 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf48011630 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.030+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581a9cf0 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.031+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fdf48011860 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.031+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5804f400 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.031+0000 7fdf4e7fc700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdf3c037780 0x7fdf3c039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.031+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fdf4804b200 con 0x7fdf5810ed90 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.031+0000 7fdf5d66f700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdf3c037780 0x7fdf3c039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:21.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.031+0000 7fdf5d66f700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdf3c037780 0x7fdf3c039c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf54000f60 tx=0x7fdf540010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:21.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.036+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fdf4802d450 con 0x7fdf5810ed90 2024-04-27T04:34:21.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.238+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["crush-failure-domain=osd", "k=3", "m=1", "name=testprofile"]} v 0) v1 -- 0x7fdf581150e0 con 0x7fdf5810ed90 2024-04-27T04:34:21.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.702+0000 7fdf4e7fc700 1 -- 172.21.15.191:0/884246 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["crush-failure-domain=osd", "k=3", "m=1", "name=testprofile"]}]=0 v17) v1 ==== 171+0+0 (secure 0 0 0) 0x7fdf48010b70 con 0x7fdf5810ed90 2024-04-27T04:34:21.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdf3c037780 msgr2=0x7fdf3c039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:21.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdf3c037780 0x7fdf3c039c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf54000f60 tx=0x7fdf540010a0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 msgr2=0x7fdf581a91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:21.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 0x7fdf581a91b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009fd0 tx=0x7fdf48004750 comp rx=0 tx=0).stop 2024-04-27T04:34:21.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 shutdown_connections 2024-04-27T04:34:21.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdf3c037780 0x7fdf3c039c40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 --2- 172.21.15.191:0/884246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5810ed90 0x7fdf581a91b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 >> 172.21.15.191:0/884246 conn(0x7fdf58070e00 msgr2=0x7fdf58110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:21.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.704+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 shutdown_connections 2024-04-27T04:34:21.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.705+0000 7fdf600d4700 1 -- 172.21.15.191:0/884246 wait complete. 2024-04-27T04:34:21.718 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.840+0000 7f1257516700 1 Processor -- start 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.840+0000 7f1257516700 1 -- start start 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.840+0000 7f1257516700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f125011a200 0x7f125011c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.840+0000 7f1257516700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125006d0f0 con 0x7f125006c510 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.840+0000 7f1257516700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125011db40 con 0x7f125011a200 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1254ab1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f125011a200 0x7f125011c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f12552b2700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f125006c510 0x7f125006c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:48342/0 (socket says 172.21.15.191:48342) 2024-04-27T04:34:21.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f12552b2700 1 -- 172.21.15.191:0/1521808465 learned_addr learned my addr 172.21.15.191:0/1521808465 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:21.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1254ab1700 1 -- 172.21.15.191:0/1521808465 >> v1:172.21.15.138:6789/0 conn(0x7f125006c510 legacy=0x7f125006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:21.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1254ab1700 1 -- 172.21.15.191:0/1521808465 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125011dc80 con 0x7f125011a200 2024-04-27T04:34:21.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1254ab1700 1 --2- 172.21.15.191:0/1521808465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125011a200 0x7f125011c5f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f12400097c0 tx=0x7f1240009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bdc9e72f2e3fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:21.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1247fff700 1 -- 172.21.15.191:0/1521808465 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1240004160 con 0x7f125011a200 2024-04-27T04:34:21.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1247fff700 1 -- 172.21.15.191:0/1521808465 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12400042c0 con 0x7f125011a200 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.841+0000 7f1247fff700 1 -- 172.21.15.191:0/1521808465 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1240004580 con 0x7f125011a200 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 -- 172.21.15.191:0/1521808465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125011a200 msgr2=0x7f125011c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 --2- 172.21.15.191:0/1521808465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125011a200 0x7f125011c5f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f12400097c0 tx=0x7f1240009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 -- 172.21.15.191:0/1521808465 shutdown_connections 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 --2- 172.21.15.191:0/1521808465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125011a200 0x7f125011c5f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 -- 172.21.15.191:0/1521808465 >> 172.21.15.191:0/1521808465 conn(0x7f1250070e00 msgr2=0x7f1250071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 -- 172.21.15.191:0/1521808465 shutdown_connections 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 -- 172.21.15.191:0/1521808465 wait complete. 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.842+0000 7f1257516700 1 Processor -- start 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1257516700 1 -- start start 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1257516700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 0x7f12501a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:21.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1257516700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a5ed0 con 0x7f125006c510 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f12552b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 0x7f12501a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f12552b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 0x7f12501a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49420/0 (socket says 172.21.15.191:49420) 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f12552b2700 1 -- 172.21.15.191:0/1228146739 learned_addr learned my addr 172.21.15.191:0/1228146739 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f12552b2700 1 -- 172.21.15.191:0/1228146739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12400094a0 con 0x7f125006c510 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f12552b2700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 0x7f12501a5990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f124c00c480 tx=0x7f124c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f124c005ca0 con 0x7f125006c510 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c005e00 con 0x7f125006c510 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f124c0049c0 con 0x7f125006c510 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.843+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12501a60d0 con 0x7f125006c510 2024-04-27T04:34:21.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.844+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12501aa0b0 con 0x7f125006c510 2024-04-27T04:34:21.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.844+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f124c004b20 con 0x7f125006c510 2024-04-27T04:34:21.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.845+0000 7f1245ffb700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f1234037790 0x7f1234039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:21.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.845+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125004f400 con 0x7f125006c510 2024-04-27T04:34:21.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.845+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3725+0+0 (secure 0 0 0) 0x7f124c0499a0 con 0x7f125006c510 2024-04-27T04:34:21.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.845+0000 7f1254ab1700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f1234037790 0x7f1234039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:21.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.845+0000 7f1254ab1700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f1234037790 0x7f1234039c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f125006dde0 tx=0x7f1240008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:21.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:21.850+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f124c011c00 con 0x7f125006c510 2024-04-27T04:34:22.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:22.041+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"} v 0) v1 -- 0x7f125011a200 con 0x7f125006c510 2024-04-27T04:34:23.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.714+0000 7f1245ffb700 1 -- 172.21.15.191:0/1228146739 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"}]=0 pool 'default.rgw.buckets.data' created v19) v1 ==== 227+0+0 (secure 0 0 0) 0x7f124c011810 con 0x7f125006c510 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.717+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f1234037790 msgr2=0x7f1234039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.717+0000 7f1257516700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f1234037790 0x7f1234039c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f125006dde0 tx=0x7f1240008030 comp rx=0 tx=0).stop 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.717+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 msgr2=0x7f12501a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.717+0000 7f1257516700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 0x7f12501a5990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f124c00c480 tx=0x7f124c00c840 comp rx=0 tx=0).stop 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.718+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 shutdown_connections 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.718+0000 7f1257516700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f1234037790 0x7f1234039c50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.718+0000 7f1257516700 1 --2- 172.21.15.191:0/1228146739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f125006c510 0x7f12501a5990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:23.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.718+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 >> 172.21.15.191:0/1228146739 conn(0x7f1250070e00 msgr2=0x7f125011c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:23.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.718+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 shutdown_connections 2024-04-27T04:34:23.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.718+0000 7f1257516700 1 -- 172.21.15.191:0/1228146739 wait complete. 2024-04-27T04:34:23.720 INFO:teuthology.orchestra.run.smithi191.stderr:pool 'default.rgw.buckets.data' created 2024-04-27T04:34:23.741 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-04-27T04:34:23.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.864+0000 7f2da72c4700 1 Processor -- start 2024-04-27T04:34:23.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.864+0000 7f2da72c4700 1 -- start start 2024-04-27T04:34:23.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.864+0000 7f2da72c4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2da01150e0 0x7f2da01154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:23.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.864+0000 7f2da72c4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da0115b50 con 0x7f2da010ed90 2024-04-27T04:34:23.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.864+0000 7f2da72c4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da010f950 con 0x7f2da01150e0 2024-04-27T04:34:23.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da5060700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2da01150e0 0x7f2da01154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:23.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da485f700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2da010ed90 0x7f2da010f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:48344/0 (socket says 172.21.15.191:48344) 2024-04-27T04:34:23.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da5060700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2da01150e0 0x7f2da01154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49430/0 (socket says 172.21.15.191:49430) 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da485f700 1 -- 172.21.15.191:0/4259053323 learned_addr learned my addr 172.21.15.191:0/4259053323 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da5060700 1 -- 172.21.15.191:0/4259053323 >> v1:172.21.15.138:6789/0 conn(0x7f2da010ed90 legacy=0x7f2da010f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da5060700 1 -- 172.21.15.191:0/4259053323 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da010fac0 con 0x7f2da01150e0 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2da5060700 1 --2- 172.21.15.191:0/4259053323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da01150e0 0x7f2da01154c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097c0 tx=0x7f2d90009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ca1ccc812cba8f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2d97fff700 1 -- 172.21.15.191:0/4259053323 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d90004160 con 0x7f2da01150e0 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2d97fff700 1 -- 172.21.15.191:0/4259053323 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d900042c0 con 0x7f2da01150e0 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.865+0000 7f2d97fff700 1 -- 172.21.15.191:0/4259053323 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d90004580 con 0x7f2da01150e0 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 -- 172.21.15.191:0/4259053323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da01150e0 msgr2=0x7f2da01154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 --2- 172.21.15.191:0/4259053323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da01150e0 0x7f2da01154c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097c0 tx=0x7f2d90009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 -- 172.21.15.191:0/4259053323 shutdown_connections 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 --2- 172.21.15.191:0/4259053323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da01150e0 0x7f2da01154c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 -- 172.21.15.191:0/4259053323 >> 172.21.15.191:0/4259053323 conn(0x7f2da0070e30 msgr2=0x7f2da0071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 -- 172.21.15.191:0/4259053323 shutdown_connections 2024-04-27T04:34:23.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.866+0000 7f2da72c4700 1 -- 172.21.15.191:0/4259053323 wait complete. 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da72c4700 1 Processor -- start 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da72c4700 1 -- start start 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da72c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 0x7f2da01a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da72c4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a9940 con 0x7f2da010ed90 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da5060700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 0x7f2da01a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da5060700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 0x7f2da01a9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49446/0 (socket says 172.21.15.191:49446) 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da5060700 1 -- 172.21.15.191:0/700937264 learned_addr learned my addr 172.21.15.191:0/700937264 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.867+0000 7f2da5060700 1 -- 172.21.15.191:0/700937264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900094a0 con 0x7f2da010ed90 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.868+0000 7f2da5060700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 0x7f2da01a9370 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009b10 tx=0x7f2d90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.868+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d90011630 con 0x7f2da010ed90 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.868+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9001e070 con 0x7f2da010ed90 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.868+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da01a9ae0 con 0x7f2da010ed90 2024-04-27T04:34:23.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.868+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d90011900 con 0x7f2da010ed90 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.868+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da01a33e0 con 0x7f2da010ed90 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.869+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da004f400 con 0x7f2da010ed90 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.869+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2d90011ab0 con 0x7f2da010ed90 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.869+0000 7f2d95ffb700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d84037740 0x7f2d84039c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.869+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4200+0+0 (secure 0 0 0) 0x7f2d9004b610 con 0x7f2da010ed90 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.869+0000 7f2da485f700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d84037740 0x7f2d84039c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:23.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.870+0000 7f2da485f700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d84037740 0x7f2d84039c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000f60 tx=0x7f2d9c007b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:23.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:23.875+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2d90010bb0 con 0x7f2da010ed90 2024-04-27T04:34:24.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.067+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) v1 -- 0x7f2da0115320 con 0x7f2da010ed90 2024-04-27T04:34:24.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.713+0000 7f2d95ffb700 1 -- 172.21.15.191:0/700937264 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v20) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2da0115320 con 0x7f2da010ed90 2024-04-27T04:34:24.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.718+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d84037740 msgr2=0x7f2d84039c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:24.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.718+0000 7f2da72c4700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d84037740 0x7f2d84039c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000f60 tx=0x7f2d9c007b10 comp rx=0 tx=0).stop 2024-04-27T04:34:24.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.718+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 msgr2=0x7f2da01a9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:24.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.718+0000 7f2da72c4700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 0x7f2da01a9370 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009b10 tx=0x7f2d90004750 comp rx=0 tx=0).stop 2024-04-27T04:34:24.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.721+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 shutdown_connections 2024-04-27T04:34:24.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.721+0000 7f2da72c4700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f2d84037740 0x7f2d84039c00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:24.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.721+0000 7f2da72c4700 1 --2- 172.21.15.191:0/700937264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da010ed90 0x7f2da01a9370 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:24.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.721+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 >> 172.21.15.191:0/700937264 conn(0x7f2da0070e30 msgr2=0x7f2da0119e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:24.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.722+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 shutdown_connections 2024-04-27T04:34:24.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.723+0000 7f2da72c4700 1 -- 172.21.15.191:0/700937264 wait complete. 2024-04-27T04:34:24.743 INFO:teuthology.orchestra.run.smithi191.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-27T04:34:24.757 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-04-27T04:34:24.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe01cb50700 1 Processor -- start 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe01cb50700 1 -- start start 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe01cb50700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0181150e0 0x7fe0181154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe01cb50700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe018115b50 con 0x7fe01810ed90 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe01cb50700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01810f950 con 0x7fe0181150e0 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe01659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0181150e0 0x7fe0181154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe015d9b700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe01810ed90 0x7fe01810f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:48352/0 (socket says 172.21.15.191:48352) 2024-04-27T04:34:24.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.886+0000 7fe015d9b700 1 -- 172.21.15.191:0/3754332855 learned_addr learned my addr 172.21.15.191:0/3754332855 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01659c700 1 -- 172.21.15.191:0/3754332855 >> v1:172.21.15.138:6789/0 conn(0x7fe01810ed90 legacy=0x7fe01810f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01659c700 1 -- 172.21.15.191:0/3754332855 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01810fac0 con 0x7fe0181150e0 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01659c700 1 --2- 172.21.15.191:0/3754332855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0181150e0 0x7fe0181154c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0097c0 tx=0x7fe00c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecc6268fd708e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01559a700 1 -- 172.21.15.191:0/3754332855 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe00c004160 con 0x7fe0181150e0 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01559a700 1 -- 172.21.15.191:0/3754332855 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c0042c0 con 0x7fe0181150e0 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01559a700 1 -- 172.21.15.191:0/3754332855 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe00c004580 con 0x7fe0181150e0 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01cb50700 1 -- 172.21.15.191:0/3754332855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0181150e0 msgr2=0x7fe0181154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.887+0000 7fe01cb50700 1 --2- 172.21.15.191:0/3754332855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0181150e0 0x7fe0181154c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0097c0 tx=0x7fe00c009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 -- 172.21.15.191:0/3754332855 shutdown_connections 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 --2- 172.21.15.191:0/3754332855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0181150e0 0x7fe0181154c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:24.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 -- 172.21.15.191:0/3754332855 >> 172.21.15.191:0/3754332855 conn(0x7fe018070e00 msgr2=0x7fe018071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 -- 172.21.15.191:0/3754332855 shutdown_connections 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 -- 172.21.15.191:0/3754332855 wait complete. 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 Processor -- start 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.888+0000 7fe01cb50700 1 -- start start 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01cb50700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 0x7fe0181a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01cb50700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0181a9880 con 0x7fe01810ed90 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 0x7fe0181a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 0x7fe0181a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49466/0 (socket says 172.21.15.191:49466) 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01659c700 1 -- 172.21.15.191:0/2187133336 learned_addr learned my addr 172.21.15.191:0/2187133336 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01659c700 1 -- 172.21.15.191:0/2187133336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c0094a0 con 0x7fe01810ed90 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01659c700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 0x7fe0181a92b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009b10 tx=0x7fe00c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe00c011630 con 0x7fe01810ed90 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c01e070 con 0x7fe01810ed90 2024-04-27T04:34:24.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe00c011900 con 0x7fe01810ed90 2024-04-27T04:34:24.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.889+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0181a9a20 con 0x7fe01810ed90 2024-04-27T04:34:24.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.890+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0181a3320 con 0x7fe01810ed90 2024-04-27T04:34:24.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.890+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe00c011a60 con 0x7fe01810ed90 2024-04-27T04:34:24.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.890+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01804f400 con 0x7fe01810ed90 2024-04-27T04:34:24.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.890+0000 7fe006ffd700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdff8037790 0x7fdff8039c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:24.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.891+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4211+0+0 (secure 0 0 0) 0x7fe00c04b5d0 con 0x7fe01810ed90 2024-04-27T04:34:24.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.891+0000 7fe015d9b700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdff8037790 0x7fdff8039c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:24.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.891+0000 7fe015d9b700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdff8037790 0x7fdff8039c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe008000f60 tx=0x7fe008007b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:24.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:24.896+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fe00c010bb0 con 0x7fe01810ed90 2024-04-27T04:34:25.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.088+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7fe0181150e0 con 0x7fe01810ed90 2024-04-27T04:34:25.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.718+0000 7fe006ffd700 1 -- 172.21.15.191:0/2187133336 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v21) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe00c0107f0 con 0x7fe01810ed90 2024-04-27T04:34:25.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdff8037790 msgr2=0x7fdff8039c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:25.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdff8037790 0x7fdff8039c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe008000f60 tx=0x7fe008007b10 comp rx=0 tx=0).stop 2024-04-27T04:34:25.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 msgr2=0x7fe0181a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:25.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 0x7fe0181a92b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009b10 tx=0x7fe00c004750 comp rx=0 tx=0).stop 2024-04-27T04:34:25.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 shutdown_connections 2024-04-27T04:34:25.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fdff8037790 0x7fdff8039c50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:25.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 --2- 172.21.15.191:0/2187133336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01810ed90 0x7fe0181a92b0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009b10 tx=0x7fe00c004750 comp rx=0 tx=0).stop 2024-04-27T04:34:25.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 >> 172.21.15.191:0/2187133336 conn(0x7fe018070e00 msgr2=0x7fe018110ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:25.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.723+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 shutdown_connections 2024-04-27T04:34:25.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.724+0000 7fe01cb50700 1 -- 172.21.15.191:0/2187133336 wait complete. 2024-04-27T04:34:25.729 INFO:teuthology.orchestra.run.smithi191.stderr:pool 'default.rgw.buckets.index' created 2024-04-27T04:34:25.744 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-04-27T04:34:25.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.865+0000 7f7c2f2c7700 1 Processor -- start 2024-04-27T04:34:25.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.865+0000 7f7c2f2c7700 1 -- start start 2024-04-27T04:34:25.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.865+0000 7f7c2f2c7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c28113ea0 0x7f7c28114280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:25.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.865+0000 7f7c2f2c7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c28114940 con 0x7f7c2810e900 2024-04-27T04:34:25.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.865+0000 7f7c2f2c7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2810f4c0 con 0x7f7c28113ea0 2024-04-27T04:34:25.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c2d063700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c28113ea0 0x7f7c28114280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:25.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c2c862700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7c2810e900 0x7f7c2810edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:48356/0 (socket says 172.21.15.191:48356) 2024-04-27T04:34:25.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c2c862700 1 -- 172.21.15.191:0/1615389765 learned_addr learned my addr 172.21.15.191:0/1615389765 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:25.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c2d063700 1 -- 172.21.15.191:0/1615389765 >> v1:172.21.15.138:6789/0 conn(0x7f7c2810e900 legacy=0x7f7c2810edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:25.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c2d063700 1 -- 172.21.15.191:0/1615389765 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2810f630 con 0x7f7c28113ea0 2024-04-27T04:34:25.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c2d063700 1 --2- 172.21.15.191:0/1615389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c28113ea0 0x7f7c28114280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c180097c0 tx=0x7f7c18009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289c851655596750 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c1ffff700 1 -- 172.21.15.191:0/1615389765 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c18004160 con 0x7f7c28113ea0 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c1ffff700 1 -- 172.21.15.191:0/1615389765 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c180042c0 con 0x7f7c28113ea0 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.866+0000 7f7c1ffff700 1 -- 172.21.15.191:0/1615389765 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c18004580 con 0x7f7c28113ea0 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/1615389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c28113ea0 msgr2=0x7f7c28114280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 --2- 172.21.15.191:0/1615389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c28113ea0 0x7f7c28114280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c180097c0 tx=0x7f7c18009ad0 comp rx=0 tx=0).stop 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/1615389765 shutdown_connections 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 --2- 172.21.15.191:0/1615389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c28113ea0 0x7f7c28114280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/1615389765 >> 172.21.15.191:0/1615389765 conn(0x7f7c2806f140 msgr2=0x7f7c28071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/1615389765 shutdown_connections 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.867+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/1615389765 wait complete. 2024-04-27T04:34:25.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2f2c7700 1 Processor -- start 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2f2c7700 1 -- start start 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2f2c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 0x7f7c281a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2f2c7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c281a47e0 con 0x7f7c2810e900 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2d063700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 0x7f7c281a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2d063700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 0x7f7c281a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49494/0 (socket says 172.21.15.191:49494) 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2d063700 1 -- 172.21.15.191:0/3262680880 learned_addr learned my addr 172.21.15.191:0/3262680880 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2d063700 1 -- 172.21.15.191:0/3262680880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180094a0 con 0x7f7c2810e900 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.868+0000 7f7c2d063700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 0x7f7c281a4210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c2810f600 tx=0x7f7c18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.869+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c18003df0 con 0x7f7c2810e900 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.869+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1801e070 con 0x7f7c2810e900 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.869+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c281a4980 con 0x7f7c2810e900 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.869+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c18011780 con 0x7f7c2810e900 2024-04-27T04:34:25.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.869+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c281a4dd0 con 0x7f7c2810e900 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.870+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c281a5080 con 0x7f7c2810e900 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.870+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7c180119b0 con 0x7f7c2810e900 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.870+0000 7f7c1dffb700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7c0c037730 0x7f7c0c039bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.870+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4575+0+0 (secure 0 0 0) 0x7f7c1804b7e0 con 0x7f7c2810e900 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.870+0000 7f7c2c862700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7c0c037730 0x7f7c0c039bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.870+0000 7f7c2c862700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7c0c037730 0x7f7c0c039bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c24000f60 tx=0x7f7c24007b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:25.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:25.876+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7c281a5080 con 0x7f7c2810e900 2024-04-27T04:34:26.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.069+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) v1 -- 0x7f7c281a5080 con 0x7f7c2810e900 2024-04-27T04:34:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.720+0000 7f7c1dffb700 1 -- 172.21.15.191:0/3262680880 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v22) v1 ==== 187+0+0 (secure 0 0 0) 0x7f7c281a5080 con 0x7f7c2810e900 2024-04-27T04:34:26.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7c0c037730 msgr2=0x7f7c0c039bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:26.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7c0c037730 0x7f7c0c039bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c24000f60 tx=0x7f7c24007b10 comp rx=0 tx=0).stop 2024-04-27T04:34:26.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 msgr2=0x7f7c281a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:26.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 0x7f7c281a4210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c2810f600 tx=0x7f7c18004750 comp rx=0 tx=0).stop 2024-04-27T04:34:26.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 shutdown_connections 2024-04-27T04:34:26.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7f7c0c037730 0x7f7c0c039bf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:26.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 --2- 172.21.15.191:0/3262680880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c2810e900 0x7f7c281a4210 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c2810f600 tx=0x7f7c18004750 comp rx=0 tx=0).stop 2024-04-27T04:34:26.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.727+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 >> 172.21.15.191:0/3262680880 conn(0x7f7c2806f140 msgr2=0x7f7c2810fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:26.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.728+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 shutdown_connections 2024-04-27T04:34:26.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:26.729+0000 7f7c2f2c7700 1 -- 172.21.15.191:0/3262680880 wait complete. 2024-04-27T04:34:26.740 INFO:teuthology.orchestra.run.smithi191.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-27T04:34:26.757 DEBUG:tasks.rgw:Pools created 2024-04-27T04:34:26.758 INFO:tasks.rgw:Starting rgw... 2024-04-27T04:34:26.758 INFO:tasks.rgw:rgw client.0 config is {} 2024-04-27T04:34:26.758 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-27T04:34:26.758 INFO:tasks.rgw.client.0:Restarting daemon 2024-04-27T04:34:26.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-04-27T04:34:26.761 INFO:tasks.rgw.client.0:Started 2024-04-27T04:34:26.761 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi191.front.sepia.ceph.com:80/ 2024-04-27T04:34:26.762 DEBUG:teuthology.orchestra.run.smithi191:> curl http://smithi191.front.sepia.ceph.com:80/ 2024-04-27T04:34:26.803 INFO:teuthology.orchestra.run.smithi191.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T04:34:26.803 INFO:teuthology.orchestra.run.smithi191.stderr: Dload Upload Total Spent Left Speed 2024-04-27T04:34:26.804 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T04:34:26.805 INFO:teuthology.orchestra.run.smithi191.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi191.front.sepia.ceph.com port 80: Connection refused 2024-04-27T04:34:27.805 DEBUG:teuthology.orchestra.run.smithi191:> curl http://smithi191.front.sepia.ceph.com:80/ 2024-04-27T04:34:27.830 INFO:teuthology.orchestra.run.smithi191.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T04:34:27.830 INFO:teuthology.orchestra.run.smithi191.stderr: Dload Upload Total Spent Left Speed 2024-04-27T04:34:27.831 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T04:34:27.832 INFO:teuthology.orchestra.run.smithi191.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi191.front.sepia.ceph.com port 80: Connection refused 2024-04-27T04:34:29.833 DEBUG:teuthology.orchestra.run.smithi191:> curl http://smithi191.front.sepia.ceph.com:80/ 2024-04-27T04:34:29.859 INFO:teuthology.orchestra.run.smithi191.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T04:34:29.859 INFO:teuthology.orchestra.run.smithi191.stderr: Dload Upload Total Spent Left Speed 2024-04-27T04:34:29.860 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T04:34:29.861 INFO:teuthology.orchestra.run.smithi191.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi191.front.sepia.ceph.com port 80: Connection refused 2024-04-27T04:34:33.862 DEBUG:teuthology.orchestra.run.smithi191:> curl http://smithi191.front.sepia.ceph.com:80/ 2024-04-27T04:34:33.888 INFO:teuthology.orchestra.run.smithi191.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T04:34:33.888 INFO:teuthology.orchestra.run.smithi191.stderr: Dload Upload Total Spent Left Speed 2024-04-27T04:34:33.889 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T04:34:33.890 INFO:teuthology.orchestra.run.smithi191.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi191.front.sepia.ceph.com port 80: Connection refused 2024-04-27T04:34:41.891 DEBUG:teuthology.orchestra.run.smithi191:> curl http://smithi191.front.sepia.ceph.com:80/ 2024-04-27T04:34:41.917 INFO:teuthology.orchestra.run.smithi191.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T04:34:41.917 INFO:teuthology.orchestra.run.smithi191.stderr: Dload Upload Total Spent Left Speed 2024-04-27T04:34:41.918 INFO:teuthology.orchestra.run.smithi191.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 208k 2024-04-27T04:34:41.919 INFO:teuthology.orchestra.run.smithi191.stdout:anonymous 2024-04-27T04:34:41.920 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-04-27T04:34:42.009 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-04-27T04:34:42.010 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-27T04:34:42.010 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-27T04:34:42.010 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-27T04:34:42.061 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.061 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f246747fa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f246747fa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fb5f31f500 con 0x55fb5eff4c00 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f246747fa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fb5f096a00 con 0x55fb5eff4800 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f2465429700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f2464c28700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55fb5eff4c00 0x55fb5f0e0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55200/0 (socket says 172.21.15.191:55200) 2024-04-27T04:34:42.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.061+0000 7f2464c28700 1 -- 172.21.15.191:0/1449821828 learned_addr learned my addr 172.21.15.191:0/1449821828 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2465429700 1 -- 172.21.15.191:0/1449821828 >> v1:172.21.15.138:6789/0 conn(0x55fb5eff4c00 legacy=0x55fb5f0e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2465429700 1 -- 172.21.15.191:0/1449821828 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01dd40 con 0x55fb5eff4800 2024-04-27T04:34:42.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2464427700 1 -- 172.21.15.191:0/1449821828 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593932246 0 0) 0x55fb5fdc2000 con 0x55fb5eff4c00 2024-04-27T04:34:42.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2465429700 1 --2- 172.21.15.191:0/1449821828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fb5f2d79e0 tx=0x55fb5f0e5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a102e02303896577 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2464427700 1 -- 172.21.15.191:0/1449821828 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad4a0 con 0x55fb5eff4800 2024-04-27T04:34:42.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2464427700 1 -- 172.21.15.191:0/1449821828 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f31f800 con 0x55fb5eff4800 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.062+0000 7f2464427700 1 -- 172.21.15.191:0/1449821828 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad600 con 0x55fb5eff4800 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 -- 172.21.15.191:0/1449821828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 msgr2=0x55fb5f2dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 --2- 172.21.15.191:0/1449821828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fb5f2d79e0 tx=0x55fb5f0e5920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 -- 172.21.15.191:0/1449821828 shutdown_connections 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 --2- 172.21.15.191:0/1449821828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 -- 172.21.15.191:0/1449821828 >> 172.21.15.191:0/1449821828 conn(0x55fb5eff4400 msgr2=0x55fb5f0e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 -- 172.21.15.191:0/1449821828 shutdown_connections 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.063+0000 7f246747fa80 1 -- 172.21.15.191:0/1449821828 wait complete. 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f246747fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f246747fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5f096500 con 0x55fb5eff4800 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f2465429700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f2465429700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37896/0 (socket says 172.21.15.191:37896) 2024-04-27T04:34:42.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f2465429700 1 -- 172.21.15.191:0/1917396967 learned_addr learned my addr 172.21.15.191:0/1917396967 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f2465429700 1 -- 172.21.15.191:0/1917396967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01dba0 con 0x55fb5eff4800 2024-04-27T04:34:42.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f2465429700 1 --2- 172.21.15.191:0/1917396967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55fb5fdc5800 tx=0x55fb5f0e5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54b94d07503731b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.064+0000 7f2463c26700 1 -- 172.21.15.191:0/1917396967 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad760 con 0x55fb5eff4800 2024-04-27T04:34:42.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f2463c26700 1 -- 172.21.15.191:0/1917396967 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f31fb00 con 0x55fb5eff4800 2024-04-27T04:34:42.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f2463c26700 1 -- 172.21.15.191:0/1917396967 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad1e0 con 0x55fb5eff4800 2024-04-27T04:34:42.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 -- 172.21.15.191:0/1917396967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 msgr2=0x55fb5f2dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 --2- 172.21.15.191:0/1917396967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55fb5fdc5800 tx=0x55fb5f0e5800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 -- 172.21.15.191:0/1917396967 shutdown_connections 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 --2- 172.21.15.191:0/1917396967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 -- 172.21.15.191:0/1917396967 >> 172.21.15.191:0/1917396967 conn(0x55fb5eff4400 msgr2=0x55fb5f0e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 -- 172.21.15.191:0/1917396967 shutdown_connections 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 -- 172.21.15.191:0/1917396967 wait complete. 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.065+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f246747fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f246747fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5f0963c0 con 0x55fb5eff4800 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f2465429700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f2465429700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37898/0 (socket says 172.21.15.191:37898) 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f2465429700 1 -- 172.21.15.191:0/1533638222 learned_addr learned my addr 172.21.15.191:0/1533638222 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f2465429700 1 -- 172.21.15.191:0/1533638222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01da00 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f2465429700 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55fb5fdc5ec0 tx=0x55fb5f0e5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.066+0000 7f2462423700 1 -- 172.21.15.191:0/1533638222 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad4a0 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f2462423700 1 -- 172.21.15.191:0/1533638222 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f31fc80 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb5f01d860 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f2462423700 1 -- 172.21.15.191:0/1533638222 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad600 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb5f01d6c0 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f246747fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.067+0000 7f246747fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5f096000 con 0x55fb5eff5400 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2465c2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2462423700 1 -- 172.21.15.191:0/1533638222 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fb5fd96b00 con 0x55fb5eff4800 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2465c2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37906/0 (socket says 172.21.15.191:37906) 2024-04-27T04:34:42.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2465c2a700 1 -- 172.21.15.191:0/759602027 learned_addr learned my addr 172.21.15.191:0/759602027 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2462423700 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38e000 0x55fb5f2dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2462423700 1 -- 172.21.15.191:0/1533638222 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55fb5f06f180 con 0x55fb5eff4800 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2465c2a700 1 -- 172.21.15.191:0/759602027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01c000 con 0x55fb5eff5400 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2465429700 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38e000 0x55fb5f2dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f2465c2a700 1 --2- 172.21.15.191:0/759602027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55fb5fe8d260 tx=0x55fb5f0e5ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103a0585e1997d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f245fc1e700 1 -- 172.21.15.191:0/759602027 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad4a0 con 0x55fb5eff5400 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f245fc1e700 1 -- 172.21.15.191:0/759602027 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f095c80 con 0x55fb5eff5400 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.068+0000 7f245fc1e700 1 -- 172.21.15.191:0/759602027 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad600 con 0x55fb5eff5400 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- 172.21.15.191:0/759602027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 msgr2=0x55fb5f2dcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 --2- 172.21.15.191:0/759602027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55fb5fe8d260 tx=0x55fb5f0e5ce0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f2465429700 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38e000 0x55fb5f2dd080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x55fb5fe8d020 tx=0x55fb5f344c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- 172.21.15.191:0/759602027 shutdown_connections 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 --2- 172.21.15.191:0/759602027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- 172.21.15.191:0/759602027 >> 172.21.15.191:0/759602027 conn(0x55fb5eff4c00 msgr2=0x55fb5f0e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- 172.21.15.191:0/759602027 shutdown_connections 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- 172.21.15.191:0/759602027 wait complete. 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f246747fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5f030640 con 0x55fb5eff5400 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f2465c2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f2465c2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37908/0 (socket says 172.21.15.191:37908) 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.069+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 learned_addr learned my addr 172.21.15.191:0/3807189627 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01cea0 con 0x55fb5eff5400 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f2465c2a700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55fb5fe8c1e0 tx=0x55fb5f0e5bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f245e41b700 1 -- 172.21.15.191:0/3807189627 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad760 con 0x55fb5eff5400 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f245e41b700 1 -- 172.21.15.191:0/3807189627 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f095b00 con 0x55fb5eff5400 2024-04-27T04:34:42.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb5f01c820 con 0x55fb5eff5400 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f245e41b700 1 -- 172.21.15.191:0/3807189627 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f0ad4a0 con 0x55fb5eff5400 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.070+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb5f01c9c0 con 0x55fb5eff5400 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.071+0000 7f245e41b700 1 -- 172.21.15.191:0/3807189627 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fb5fd96dc0 con 0x55fb5eff5400 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.071+0000 7f245e41b700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38ec00 0x55fb5f2dd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.071+0000 7f245e41b700 1 -- 172.21.15.191:0/3807189627 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55fb5f06f340 con 0x55fb5eff5400 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.071+0000 7f2464c28700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38ec00 0x55fb5f2dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f2464c28700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38ec00 0x55fb5f2dd600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x55fb5fdc4030 tx=0x55fb5f4280c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fb5eff5800 0x55fb5f2ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5eff5c00 con 0x55fb5eff5800 2024-04-27T04:34:42.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f2465c2a700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fb5eff5800 0x55fb5f2ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f2465c2a700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fb5eff5800 0x55fb5f2ddb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 20 realm 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.072+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f000 con 0x55fb5eff5800 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f400 con 0x55fb5eff5800 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f800 con 0x55fb5eff5800 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fb5f38fc00 0x55fb5f2de100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f000 con 0x55fb5f38fc00 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.073+0000 7f2465429700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fb5f38fc00 0x55fb5f2de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f2465429700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fb5f38fc00 0x55fb5f2de100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f246747fa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f246747fa80 20 searching for the correct realm 2024-04-27T04:34:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f400 con 0x55fb5f38fc00 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f2465429700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55fb5f38f400 0x55fb5f2dec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.074+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f000 con 0x55fb5f38f400 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.075+0000 7f2464c28700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55fb5f38f400 0x55fb5f2dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.075+0000 7f2464c28700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55fb5f38f400 0x55fb5f2dec00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.075+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.075+0000 7f2464c28700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55fb5f38f000 0x55fb5f2df180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.075+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.076+0000 7f2465c2a700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55fb5f38f000 0x55fb5f2df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.076+0000 7f2465c2a700 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55fb5f38f000 0x55fb5f2df180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.076+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.076+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.077+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f400 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.078+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.078+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.078+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.078+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.078+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.078+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f400 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.079+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.080+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.080+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.080+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.080+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.080+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.081+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.082+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.082+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f400 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.082+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.082+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.082+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.082+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f400 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.083+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38fc00 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.084+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.084+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.084+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.084+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.084+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.084+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f400 2024-04-27T04:34:42.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5eff5c00 con 0x55fb5eff5800 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446400 con 0x55fb5eff5800 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.085+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446800 con 0x55fb5f38fc00 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 zone default found 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 4 Realm: () 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 zonegroup default 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446c00 con 0x55fb5f38f000 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447000 con 0x55fb5f38fc00 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.086+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447400 con 0x55fb5f38fc00 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.087+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.087+0000 7f246747fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.087+0000 7f246747fa80 20 started sync module instance, tier type = 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.087+0000 7f246747fa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.087+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.088+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.088+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94538121952000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446800 con 0x55fb5f38f400 2024-04-27T04:34:42.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.088+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447000 con 0x55fb5eff5800 2024-04-27T04:34:42.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.089+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94538121952000] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.089+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.089+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94538121953152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447400 con 0x55fb5eff5800 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.089+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446400 con 0x55fb5f38fc00 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.091+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94538121953152] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.093+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.093+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94538121954304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446c00 con 0x55fb5f38fc00 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.093+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5eff5c00 con 0x55fb5f38f400 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.094+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94538121954304] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.094+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.094+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94538121955456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f38f800 con 0x55fb5f38f400 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.094+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5eff5800 2024-04-27T04:34:42.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.097+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94538121955456] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.100+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.100+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94538121956608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.100+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446800 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.101+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.101+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94538121957760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447c00 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.101+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447000 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.101+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94538121956608] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.102+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94538121957760] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.102+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.102+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94538121958912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447400 con 0x55fb5f38f000 2024-04-27T04:34:42.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.102+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446000 con 0x55fb5f38f000 2024-04-27T04:34:42.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94538121958912] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94538121960064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=0 2024-04-27T04:34:42.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=1 2024-04-27T04:34:42.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=2 2024-04-27T04:34:42.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=3 2024-04-27T04:34:42.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=4 2024-04-27T04:34:42.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=5 2024-04-27T04:34:42.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.103+0000 7f246747fa80 20 add_watcher() i=6 2024-04-27T04:34:42.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.104+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94538121960064] v31'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.104+0000 7f246747fa80 20 add_watcher() i=7 2024-04-27T04:34:42.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.104+0000 7f246747fa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:42.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.104+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446800 con 0x55fb5eff5800 2024-04-27T04:34:42.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.105+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.105+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94538123517952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447c00 con 0x55fb5eff5800 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94538123517952] v31'30 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f244c3f7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f246747fa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f246747fa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.107+0000 7f246747fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.108+0000 7f246747fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5f030780 con 0x55fb5f447400 2024-04-27T04:34:42.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.108+0000 7f2464c28700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.108+0000 7f2464c28700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37910/0 (socket says 172.21.15.191:37910) 2024-04-27T04:34:42.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.108+0000 7f2464c28700 1 -- 172.21.15.191:0/3649991012 learned_addr learned my addr 172.21.15.191:0/3649991012 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.108+0000 7f2464c28700 1 -- 172.21.15.191:0/3649991012 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01cb60 con 0x55fb5f447400 2024-04-27T04:34:42.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f2464c28700 1 --2- 172.21.15.191:0/3649991012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55fb5f3da480 tx=0x55fb5f345f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a0b65c51c9dc61 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f244bbf6700 1 -- 172.21.15.191:0/3649991012 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f464000 con 0x55fb5f447400 2024-04-27T04:34:42.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f244bbf6700 1 -- 172.21.15.191:0/3649991012 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f468000 con 0x55fb5f447400 2024-04-27T04:34:42.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f244bbf6700 1 -- 172.21.15.191:0/3649991012 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f464160 con 0x55fb5f447400 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f246747fa80 1 -- 172.21.15.191:0/3649991012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 msgr2=0x55fb5f2de680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f246747fa80 1 --2- 172.21.15.191:0/3649991012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55fb5f3da480 tx=0x55fb5f345f80 comp rx=0 tx=0).stop 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f246747fa80 1 -- 172.21.15.191:0/3649991012 shutdown_connections 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f246747fa80 1 --2- 172.21.15.191:0/3649991012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f246747fa80 1 -- 172.21.15.191:0/3649991012 >> 172.21.15.191:0/3649991012 conn(0x55fb5f447000 msgr2=0x55fb5f0e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.109+0000 7f246747fa80 1 -- 172.21.15.191:0/3649991012 shutdown_connections 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f246747fa80 1 -- 172.21.15.191:0/3649991012 wait complete. 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f246747fa80 1 Processor -- start 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f246747fa80 1 -- start start 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f246747fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f246747fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5f096000 con 0x55fb5f447400 2024-04-27T04:34:42.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f2464c28700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f2464c28700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37918/0 (socket says 172.21.15.191:37918) 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.110+0000 7f2464c28700 1 -- 172.21.15.191:0/638136713 learned_addr learned my addr 172.21.15.191:0/638136713 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f2464c28700 1 -- 172.21.15.191:0/638136713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5f01cd00 con 0x55fb5f447400 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f2464c28700 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55fb5f467200 tx=0x55fb5f4284e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f244a3f3700 1 -- 172.21.15.191:0/638136713 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f4642c0 con 0x55fb5f447400 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f244a3f3700 1 -- 172.21.15.191:0/638136713 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5f468180 con 0x55fb5f447400 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb5f01c340 con 0x55fb5f447400 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f244a3f3700 1 -- 172.21.15.191:0/638136713 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb5f464000 con 0x55fb5f447400 2024-04-27T04:34:42.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.111+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb5f01cb60 con 0x55fb5f447400 2024-04-27T04:34:42.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.112+0000 7f244a3f3700 1 -- 172.21.15.191:0/638136713 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fb5fd97b80 con 0x55fb5f447400 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.112+0000 7f244a3f3700 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f4e8000 0x55fb5f2df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.112+0000 7f244a3f3700 1 -- 172.21.15.191:0/638136713 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55fb5f06f500 con 0x55fb5f447400 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.112+0000 7f246747fa80 5 note: GC not initialized 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.112+0000 7f2465c2a700 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f4e8000 0x55fb5f2df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.112+0000 7f2465c2a700 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f4e8000 0x55fb5f2df700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55fb5f3db800 tx=0x55fb5f428360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.113+0000 7f2447bee700 20 reqs_thread_entry: start 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.113+0000 7f246747fa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.113+0000 7f246747fa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.113+0000 7f2445bea700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:12 2024 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.113+0000 7f24453c8700 20 reqs_thread_entry: start 2024-04-27T04:34:42.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.113+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.114+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(62 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.114+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.114+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(63 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.114+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.114+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.114+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(66 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.115+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(69 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(70 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(71 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(72 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.116+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.117+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(73 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.117+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.117+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.117+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.117+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.117+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(76 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(77 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(78 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(79 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.118+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(80 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(81 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(82 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(83 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(84 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.119+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.120+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(85 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.120+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.120+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(86 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.120+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.120+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(87 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.120+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.121+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(88 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.121+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.121+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(89 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.121+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f000 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.121+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(90 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.121+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5eff5800 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.122+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(91 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.122+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38fc00 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.122+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(92 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.122+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f447800 con 0x55fb5f38f400 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.122+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(93 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.123+0000 7f244c3f7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.123+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94538123517952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f446800 con 0x55fb5eff5800 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.124+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94538123517952] v31'31 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.124+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94538121952000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e8400 con 0x55fb5f38f400 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.125+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94538121952000] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.125+0000 7f246747fa80 20 remove_watcher() i=0 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.125+0000 7f246747fa80 2 removed watcher, disabling cache 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.125+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94538121953152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e8800 con 0x55fb5eff5800 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.126+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94538121953152] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.126+0000 7f246747fa80 20 remove_watcher() i=1 2024-04-27T04:34:42.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.126+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94538121954304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e8c00 con 0x55fb5f38fc00 2024-04-27T04:34:42.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.127+0000 7f2465429700 1 -- 172.21.15.191:0/3807189627 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94538121954304] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f200 con 0x55fb5f38fc00 2024-04-27T04:34:42.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.127+0000 7f246747fa80 20 remove_watcher() i=2 2024-04-27T04:34:42.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.127+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94538121955456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e9000 con 0x55fb5f38f400 2024-04-27T04:34:42.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.128+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94538121955456] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43f440 con 0x55fb5f38f400 2024-04-27T04:34:42.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.128+0000 7f246747fa80 20 remove_watcher() i=3 2024-04-27T04:34:42.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.128+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94538121956608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e9400 con 0x55fb5eff5800 2024-04-27T04:34:42.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.130+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94538121956608] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5eff5800 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.130+0000 7f246747fa80 20 remove_watcher() i=4 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.130+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94538121957760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e9800 con 0x55fb5f38f000 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.131+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94538121957760] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.131+0000 7f246747fa80 20 remove_watcher() i=5 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.131+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94538121958912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e9c00 con 0x55fb5f38f000 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.133+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94538121958912] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.133+0000 7f246747fa80 20 remove_watcher() i=6 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.133+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94538121960064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb5f4e8400 con 0x55fb5f38f000 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f2465c2a700 1 -- 172.21.15.191:0/3807189627 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94538121960064] v31'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb5f43eb40 con 0x55fb5f38f000 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 20 remove_watcher() i=7 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55fb5f38f000 msgr2=0x55fb5f2df180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55fb5f38f000 0x55fb5f2df180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fb5eff5800 msgr2=0x55fb5f2ddb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fb5eff5800 0x55fb5f2ddb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55fb5f38f400 msgr2=0x55fb5f2dec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55fb5f38f400 0x55fb5f2dec00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fb5f38fc00 msgr2=0x55fb5f2de100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fb5f38fc00 0x55fb5f2de100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38ec00 msgr2=0x55fb5f2dd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38ec00 0x55fb5f2dd600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x55fb5fdc4030 tx=0x55fb5f4280c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 msgr2=0x55fb5f2dcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 --2- 172.21.15.191:0/3807189627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff5400 0x55fb5f2dcb00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55fb5fe8c1e0 tx=0x55fb5f0e5bc0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 reap_dead start 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f2464c28700 1 -- 172.21.15.191:0/3807189627 reap_dead start 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 shutdown_connections 2024-04-27T04:34:42.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 >> 172.21.15.191:0/3807189627 conn(0x55fb5eff4c00 msgr2=0x55fb5f0e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 shutdown_connections 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.134+0000 7f246747fa80 1 -- 172.21.15.191:0/3807189627 wait complete. 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f4e8000 msgr2=0x55fb5f2df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f4e8000 0x55fb5f2df700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55fb5f3db800 tx=0x55fb5f428360 comp rx=0 tx=0).stop 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 msgr2=0x55fb5f2de680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55fb5f467200 tx=0x55fb5f4284e0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 shutdown_connections 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f4e8000 0x55fb5f2df700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/638136713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5f447400 0x55fb5f2de680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 >> 172.21.15.191:0/638136713 conn(0x55fb5f447000 msgr2=0x55fb5f0e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 shutdown_connections 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/638136713 wait complete. 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38e000 msgr2=0x55fb5f2dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38e000 0x55fb5f2dd080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x55fb5fe8d020 tx=0x55fb5f344c60 comp rx=0 tx=0).stop 2024-04-27T04:34:42.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 msgr2=0x55fb5f2dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55fb5fdc5ec0 tx=0x55fb5f0e5920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 shutdown_connections 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fb5f38e000 0x55fb5f2dd080 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 --2- 172.21.15.191:0/1533638222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb5eff4800 0x55fb5f2dc580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 >> 172.21.15.191:0/1533638222 conn(0x55fb5eff4400 msgr2=0x55fb5f0e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 shutdown_connections 2024-04-27T04:34:42.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.135+0000 7f246747fa80 1 -- 172.21.15.191:0/1533638222 wait complete. 2024-04-27T04:34:42.142 INFO:teuthology.orchestra.run.smithi191.stdout:[] 2024-04-27T04:34:42.142 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T04:34:42.142 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-04-27T04:34:42.142 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-04-27T04:34:42.142 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-04-27T04:34:42.188 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.188 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.187+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.187+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.187+0000 7f514b944a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.187+0000 7f514b944a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55dc846e5500 con 0x55dc843bac00 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.187+0000 7f514b944a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55dc8445ca00 con 0x55dc843ba800 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51498ee700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51490ed700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55dc843bac00 0x55dc844a6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55208/0 (socket says 172.21.15.191:55208) 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51490ed700 1 -- 172.21.15.191:0/1901841504 learned_addr learned my addr 172.21.15.191:0/1901841504 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51488ec700 1 -- 172.21.15.191:0/1901841504 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250776677 0 0) 0x55dc85188000 con 0x55dc843bac00 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51488ec700 1 -- 172.21.15.191:0/1901841504 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55dc846e5680 con 0x55dc843bac00 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51498ee700 1 -- 172.21.15.191:0/1901841504 >> v1:172.21.15.138:6789/0 conn(0x55dc843bac00 legacy=0x55dc844a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51498ee700 1 -- 172.21.15.191:0/1901841504 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e3d40 con 0x55dc843ba800 2024-04-27T04:34:42.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51498ee700 1 --2- 172.21.15.191:0/1901841504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55dc8469d9e0 tx=0x55dc844ab980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85047c2e3ed9c348 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51488ec700 1 -- 172.21.15.191:0/1901841504 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc844734a0 con 0x55dc843ba800 2024-04-27T04:34:42.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51488ec700 1 -- 172.21.15.191:0/1901841504 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc846e5500 con 0x55dc843ba800 2024-04-27T04:34:42.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.188+0000 7f51488ec700 1 -- 172.21.15.191:0/1901841504 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc84473600 con 0x55dc843ba800 2024-04-27T04:34:42.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 -- 172.21.15.191:0/1901841504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 msgr2=0x55dc846a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 --2- 172.21.15.191:0/1901841504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55dc8469d9e0 tx=0x55dc844ab980 comp rx=0 tx=0).stop 2024-04-27T04:34:42.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 -- 172.21.15.191:0/1901841504 shutdown_connections 2024-04-27T04:34:42.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 --2- 172.21.15.191:0/1901841504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 -- 172.21.15.191:0/1901841504 >> 172.21.15.191:0/1901841504 conn(0x55dc843ba400 msgr2=0x55dc844a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 -- 172.21.15.191:0/1901841504 shutdown_connections 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.189+0000 7f514b944a80 1 -- 172.21.15.191:0/1901841504 wait complete. 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f514b944a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f514b944a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8445c500 con 0x55dc843ba800 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51498ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51498ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37944/0 (socket says 172.21.15.191:37944) 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51498ee700 1 -- 172.21.15.191:0/1895360337 learned_addr learned my addr 172.21.15.191:0/1895360337 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51498ee700 1 -- 172.21.15.191:0/1895360337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e3ba0 con 0x55dc843ba800 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51498ee700 1 --2- 172.21.15.191:0/1895360337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55dc8518b800 tx=0x55dc844ab800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76657e73a90dab39 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51480eb700 1 -- 172.21.15.191:0/1895360337 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc84473760 con 0x55dc843ba800 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51480eb700 1 -- 172.21.15.191:0/1895360337 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc846e5980 con 0x55dc843ba800 2024-04-27T04:34:42.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.190+0000 7f51480eb700 1 -- 172.21.15.191:0/1895360337 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc844734a0 con 0x55dc843ba800 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- 172.21.15.191:0/1895360337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 msgr2=0x55dc846a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 --2- 172.21.15.191:0/1895360337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55dc8518b800 tx=0x55dc844ab800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- 172.21.15.191:0/1895360337 shutdown_connections 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 --2- 172.21.15.191:0/1895360337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- 172.21.15.191:0/1895360337 >> 172.21.15.191:0/1895360337 conn(0x55dc843ba400 msgr2=0x55dc844a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- 172.21.15.191:0/1895360337 shutdown_connections 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- 172.21.15.191:0/1895360337 wait complete. 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.191+0000 7f514b944a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8445c3c0 con 0x55dc843ba800 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51498ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51498ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37950/0 (socket says 172.21.15.191:37950) 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51498ee700 1 -- 172.21.15.191:0/3871726645 learned_addr learned my addr 172.21.15.191:0/3871726645 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51498ee700 1 -- 172.21.15.191:0/3871726645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e3a00 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51498ee700 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55dc8518bec0 tx=0x55dc844ab980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51468e8700 1 -- 172.21.15.191:0/3871726645 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc84473600 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51468e8700 1 -- 172.21.15.191:0/3871726645 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc846e5b00 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f51468e8700 1 -- 172.21.15.191:0/3871726645 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc844738c0 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dc843e3860 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.192+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc843e36c0 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514b944a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514b944a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc843f6640 con 0x55dc843bb400 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f51468e8700 1 -- 172.21.15.191:0/3871726645 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55dc8515cb00 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f51468e8700 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474c000 0x55dc846a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f51468e8700 1 -- 172.21.15.191:0/3871726645 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55dc84435180 con 0x55dc843ba800 2024-04-27T04:34:42.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514a0ef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514a0ef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37952/0 (socket says 172.21.15.191:37952) 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514a0ef700 1 -- 172.21.15.191:0/1673972248 learned_addr learned my addr 172.21.15.191:0/1673972248 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f51498ee700 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474c000 0x55dc846a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f514a0ef700 1 -- 172.21.15.191:0/1673972248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e2000 con 0x55dc843bb400 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.193+0000 7f51498ee700 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474c000 0x55dc846a3080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55dc8523d080 tx=0x55dc84712b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1673972248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55dc8523c900 tx=0x55dc84712e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c78b906a1c5fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f51440e3700 1 -- 172.21.15.191:0/1673972248 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc84473600 con 0x55dc843bb400 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f51440e3700 1 -- 172.21.15.191:0/1673972248 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc8445bc80 con 0x55dc843bb400 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f51440e3700 1 -- 172.21.15.191:0/1673972248 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc844738c0 con 0x55dc843bb400 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 -- 172.21.15.191:0/1673972248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 msgr2=0x55dc846a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 --2- 172.21.15.191:0/1673972248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55dc8523c900 tx=0x55dc84712e40 comp rx=0 tx=0).stop 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 -- 172.21.15.191:0/1673972248 shutdown_connections 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 --2- 172.21.15.191:0/1673972248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 -- 172.21.15.191:0/1673972248 >> 172.21.15.191:0/1673972248 conn(0x55dc843bac00 msgr2=0x55dc844a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 -- 172.21.15.191:0/1673972248 shutdown_connections 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 -- 172.21.15.191:0/1673972248 wait complete. 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.194+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514b944a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514b944a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc843f68c0 con 0x55dc843bb400 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514a0ef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514a0ef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37964/0 (socket says 172.21.15.191:37964) 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 learned_addr learned my addr 172.21.15.191:0/1316101019 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e2ea0 con 0x55dc843bb400 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55dc8523c930 tx=0x55dc844abce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f51428e0700 1 -- 172.21.15.191:0/1316101019 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc84473760 con 0x55dc843bb400 2024-04-27T04:34:42.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f51428e0700 1 -- 172.21.15.191:0/1316101019 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc8445bb00 con 0x55dc843bb400 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dc843e2820 con 0x55dc843bb400 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f51428e0700 1 -- 172.21.15.191:0/1316101019 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc84473600 con 0x55dc843bb400 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.195+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc843e29c0 con 0x55dc843bb400 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.196+0000 7f51428e0700 1 -- 172.21.15.191:0/1316101019 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55dc8515cdc0 con 0x55dc843bb400 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.196+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.196+0000 7f51428e0700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474cc00 0x55dc846a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.196+0000 7f51428e0700 1 -- 172.21.15.191:0/1316101019 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55dc84435340 con 0x55dc843bb400 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.196+0000 7f51490ed700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474cc00 0x55dc846a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.196+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc843bb800 0x55dc846a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc843bbc00 con 0x55dc843bb800 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc843bb800 0x55dc846a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f51490ed700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474cc00 0x55dc846a3600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55dc8518a030 tx=0x55dc847fc0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc843bb800 0x55dc846a3b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 20 realm 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d000 con 0x55dc843bb800 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.197+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d400 con 0x55dc843bb800 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d800 con 0x55dc843bb800 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc8474dc00 0x55dc846a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d000 con 0x55dc8474dc00 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f51498ee700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc8474dc00 0x55dc846a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.198+0000 7f51498ee700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc8474dc00 0x55dc846a4100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f514b944a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f514b944a80 20 searching for the correct realm 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d400 con 0x55dc8474dc00 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f51498ee700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc8474d400 0x55dc846a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d000 con 0x55dc8474d400 2024-04-27T04:34:42.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f51490ed700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc8474d400 0x55dc846a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.199+0000 7f51490ed700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc8474d400 0x55dc846a4c00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.200+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.200+0000 7f51490ed700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8474d000 0x55dc846a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.200+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.200+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8474d000 0x55dc846a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.200+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8474d000 0x55dc846a5180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.201+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d400 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.202+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d400 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.203+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.204+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d400 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.205+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.206+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d400 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474dc00 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.207+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d400 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:42.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc843bbc00 con 0x55dc843bb800 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c400 con 0x55dc843bb800 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c800 con 0x55dc8474dc00 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.208+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 zone default found 2024-04-27T04:34:42.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 4 Realm: () 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 zonegroup default 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480cc00 con 0x55dc8474d000 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d000 con 0x55dc8474dc00 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d400 con 0x55dc8474dc00 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 started sync module instance, tier type = 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.209+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc8474d400 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.210+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.210+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94405602679552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c800 con 0x55dc8474d400 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.210+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d000 con 0x55dc843bb800 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.213+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.213+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94405602680704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d400 con 0x55dc843bb800 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.213+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c400 con 0x55dc8474dc00 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.213+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94405602679552] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.216+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94405602680704] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.216+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.216+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94405602681856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480cc00 con 0x55dc8474dc00 2024-04-27T04:34:42.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.216+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc843bbc00 con 0x55dc8474d400 2024-04-27T04:34:42.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.218+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94405602681856] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.218+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.218+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94405602683008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8474d800 con 0x55dc8474d400 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.218+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc843bb800 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.221+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.221+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94405602684160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc843bb800 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.221+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c800 con 0x55dc8474d000 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.224+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94405602683008] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.224+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.224+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94405602685312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480dc00 con 0x55dc8474d000 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.224+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d000 con 0x55dc8474d000 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.226+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94405602684160] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.226+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.226+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94405602686464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d400 con 0x55dc8474d000 2024-04-27T04:34:42.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.227+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c000 con 0x55dc8474d000 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94405602685312] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94405602687616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc8474d000 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 20 add_watcher() i=0 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 20 add_watcher() i=1 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 20 add_watcher() i=2 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 20 add_watcher() i=3 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 20 add_watcher() i=4 2024-04-27T04:34:42.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.229+0000 7f514b944a80 20 add_watcher() i=5 2024-04-27T04:34:42.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94405602686464] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514b944a80 20 add_watcher() i=6 2024-04-27T04:34:42.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94405602687616] v31'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514b944a80 20 add_watcher() i=7 2024-04-27T04:34:42.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514b944a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:42.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c800 con 0x55dc843bb800 2024-04-27T04:34:42.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.232+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94405604245504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480dc00 con 0x55dc843bb800 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.234+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94405604245504] v31'32 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.234+0000 7f51308bc700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.234+0000 7f514b944a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.234+0000 7f514b944a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.234+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f514b944a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f514b944a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8445c3c0 con 0x55dc8480d400 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f51490ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f51490ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37978/0 (socket says 172.21.15.191:37978) 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f51490ed700 1 -- 172.21.15.191:0/3607423260 learned_addr learned my addr 172.21.15.191:0/3607423260 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f51490ed700 1 -- 172.21.15.191:0/3607423260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e2b60 con 0x55dc8480d400 2024-04-27T04:34:42.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f51490ed700 1 --2- 172.21.15.191:0/3607423260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55dc847a9800 tx=0x55dc84713f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713bdf215e7e6094 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.235+0000 7f51300bb700 1 -- 172.21.15.191:0/3607423260 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc8482a000 con 0x55dc8480d400 2024-04-27T04:34:42.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f51300bb700 1 -- 172.21.15.191:0/3607423260 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc84822180 con 0x55dc8480d400 2024-04-27T04:34:42.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f51300bb700 1 -- 172.21.15.191:0/3607423260 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc8482a160 con 0x55dc8480d400 2024-04-27T04:34:42.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 -- 172.21.15.191:0/3607423260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 msgr2=0x55dc846a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 --2- 172.21.15.191:0/3607423260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55dc847a9800 tx=0x55dc84713f80 comp rx=0 tx=0).stop 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 -- 172.21.15.191:0/3607423260 shutdown_connections 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 --2- 172.21.15.191:0/3607423260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 -- 172.21.15.191:0/3607423260 >> 172.21.15.191:0/3607423260 conn(0x55dc8480d000 msgr2=0x55dc844a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 -- 172.21.15.191:0/3607423260 shutdown_connections 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 -- 172.21.15.191:0/3607423260 wait complete. 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.236+0000 7f514b944a80 1 Processor -- start 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f514b944a80 1 -- start start 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f514b944a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f514b944a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8445c500 con 0x55dc8480d400 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f51490ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f51490ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:37992/0 (socket says 172.21.15.191:37992) 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 learned_addr learned my addr 172.21.15.191:0/1735810487 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc843e2d00 con 0x55dc8480d400 2024-04-27T04:34:42.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.237+0000 7f51490ed700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55dc8481d200 tx=0x55dc847fc4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f512e8b8700 1 -- 172.21.15.191:0/1735810487 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc8482a2c0 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f512e8b8700 1 -- 172.21.15.191:0/1735810487 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc84822300 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dc843e2340 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f512e8b8700 1 -- 172.21.15.191:0/1735810487 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dc8482a000 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc843e2b60 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f512e8b8700 1 -- 172.21.15.191:0/1735810487 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55dc8515db80 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f512e8b8700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc848ae000 0x55dc846a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.238+0000 7f512e8b8700 1 -- 172.21.15.191:0/1735810487 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55dc84435500 con 0x55dc8480d400 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.239+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc848ae000 0x55dc846a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.239+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc8480c000 0x55dc848c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.239+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc8480c000 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.239+0000 7f51498ee700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc8480c000 0x55dc848c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.239+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc848ae000 0x55dc846a5700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55dc847a94a0 tx=0x55dc847fc360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.239+0000 7f51498ee700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc8480c000 0x55dc848c4000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.240+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'40 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8480c000 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.240+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8480c800 0x55dc848c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.240+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848ae400 con 0x55dc8480c800 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.240+0000 7f51490ed700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8480c800 0x55dc848c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.241+0000 7f51490ed700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8480c800 0x55dc848c4580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.242+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'16 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.242+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc848ae800 0x55dc848c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.242+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848aec00 con 0x55dc848ae800 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.242+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc848ae800 0x55dc848c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.242+0000 7f514a0ef700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc848ae800 0x55dc848c4b00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.243+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.243+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848af000 con 0x55dc8480c000 2024-04-27T04:34:42.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.244+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'33 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8480c000 2024-04-27T04:34:42.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.244+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc848af400 0x55dc848c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.244+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848af800 con 0x55dc848af400 2024-04-27T04:34:42.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.244+0000 7f51498ee700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc848af400 0x55dc848c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.245+0000 7f51498ee700 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc848af400 0x55dc848c5080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.245+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'26 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.246+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848afc00 con 0x55dc848ae800 2024-04-27T04:34:42.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.246+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'25 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.246+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848af000 con 0x55dc8480c000 2024-04-27T04:34:42.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.247+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8480c000 2024-04-27T04:34:42.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.248+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c400 con 0x55dc8480c800 2024-04-27T04:34:42.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.249+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'21 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.249+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480cc00 con 0x55dc848af400 2024-04-27T04:34:42.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.249+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.250+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc848ae800 2024-04-27T04:34:42.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.250+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.250+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6000 con 0x55dc848af400 2024-04-27T04:34:42.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.251+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'27 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.251+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6400 con 0x55dc8480c800 2024-04-27T04:34:42.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.252+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.252+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6800 con 0x55dc848af400 2024-04-27T04:34:42.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.253+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.253+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6c00 con 0x55dc8480c800 2024-04-27T04:34:42.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.254+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'31 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.254+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7000 con 0x55dc8480c000 2024-04-27T04:34:42.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.255+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'34 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8480c000 2024-04-27T04:34:42.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.255+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7400 con 0x55dc848ae800 2024-04-27T04:34:42.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.257+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.257+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7800 con 0x55dc848af400 2024-04-27T04:34:42.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.258+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'28 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.258+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7c00 con 0x55dc848ae800 2024-04-27T04:34:42.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.258+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'22 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.258+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6000 con 0x55dc8480c000 2024-04-27T04:34:42.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.260+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8480c000 2024-04-27T04:34:42.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.260+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6800 con 0x55dc8480c000 2024-04-27T04:34:42.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.261+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'35 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8480c000 2024-04-27T04:34:42.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.261+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6400 con 0x55dc848ae800 2024-04-27T04:34:42.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.262+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'21 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.262+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480cc00 con 0x55dc848af400 2024-04-27T04:34:42.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.262+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'26 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.263+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480c400 con 0x55dc848ae800 2024-04-27T04:34:42.265 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.264+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.265 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.264+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848af800 con 0x55dc848ae800 2024-04-27T04:34:42.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.267+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'28 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.267+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848af000 con 0x55dc848ae800 2024-04-27T04:34:42.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.270+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.270+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848ae400 con 0x55dc848ae800 2024-04-27T04:34:42.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.272+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'26 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.272+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc843bbc00 con 0x55dc8480c800 2024-04-27T04:34:42.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.275+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.275+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7400 con 0x55dc848ae800 2024-04-27T04:34:42.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.278+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'22 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.278+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7c00 con 0x55dc8480c800 2024-04-27T04:34:42.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.281+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'22 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.281+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6400 con 0x55dc848ae800 2024-04-27T04:34:42.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.282+0000 7f514a0ef700 1 -- 172.21.15.191:0/1735810487 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'22 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc848ae800 2024-04-27T04:34:42.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.282+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480dc00 con 0x55dc8480c800 2024-04-27T04:34:42.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.283+0000 7f51490ed700 1 -- 172.21.15.191:0/1735810487 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'32 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8480c800 2024-04-27T04:34:42.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.283+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc848af400 2024-04-27T04:34:42.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.284+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc848af400 2024-04-27T04:34:42.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f512c0b3700 20 reqs_thread_entry: start 2024-04-27T04:34:42.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f514b944a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:42.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f514b944a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f512a0af700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:12 2024 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f512988d700 20 reqs_thread_entry: start 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f514b944a80 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f514b944a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.285+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848afc00 con 0x55dc8474d400 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.286+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(62 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.286+0000 7f514b944a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.286+0000 7f514b944a80 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: 2024-04-27T04:34:42.286+0000 7f514b944a80 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:unable to remove user, user does not exist 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.286+0000 7f51308bc700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.286+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94405604245504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480dc00 con 0x55dc843bb800 2024-04-27T04:34:42.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.287+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94405604245504] v31'36 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.287+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94405602679552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480cc00 con 0x55dc8474d400 2024-04-27T04:34:42.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.288+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94405602679552] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.288+0000 7f514b944a80 20 remove_watcher() i=0 2024-04-27T04:34:42.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.288+0000 7f514b944a80 2 removed watcher, disabling cache 2024-04-27T04:34:42.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.288+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94405602680704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc8480d800 con 0x55dc843bb800 2024-04-27T04:34:42.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.289+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94405602680704] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.289+0000 7f514b944a80 20 remove_watcher() i=1 2024-04-27T04:34:42.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.289+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94405602681856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6c00 con 0x55dc8474dc00 2024-04-27T04:34:42.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.290+0000 7f51498ee700 1 -- 172.21.15.191:0/1316101019 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94405602681856] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff200 con 0x55dc8474dc00 2024-04-27T04:34:42.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.291+0000 7f514b944a80 20 remove_watcher() i=2 2024-04-27T04:34:42.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.291+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94405602683008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7c00 con 0x55dc8474d400 2024-04-27T04:34:42.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.291+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94405602683008] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847ff440 con 0x55dc8474d400 2024-04-27T04:34:42.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.291+0000 7f514b944a80 20 remove_watcher() i=3 2024-04-27T04:34:42.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.291+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94405602684160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7000 con 0x55dc843bb800 2024-04-27T04:34:42.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.292+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94405602684160] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc843bb800 2024-04-27T04:34:42.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.292+0000 7f514b944a80 20 remove_watcher() i=4 2024-04-27T04:34:42.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.292+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94405602685312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f7800 con 0x55dc8474d000 2024-04-27T04:34:42.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.293+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94405602685312] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.293+0000 7f514b944a80 20 remove_watcher() i=5 2024-04-27T04:34:42.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.293+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94405602686464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6000 con 0x55dc8474d000 2024-04-27T04:34:42.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.294+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94405602686464] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.294+0000 7f514b944a80 20 remove_watcher() i=6 2024-04-27T04:34:42.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.294+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94405602687616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dc848f6800 con 0x55dc8474d000 2024-04-27T04:34:42.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514a0ef700 1 -- 172.21.15.191:0/1316101019 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94405602687616] v31'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dc847feb40 con 0x55dc8474d000 2024-04-27T04:34:42.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 20 remove_watcher() i=7 2024-04-27T04:34:42.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8474d000 msgr2=0x55dc846a5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8474d000 0x55dc846a5180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc843bb800 msgr2=0x55dc846a3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc843bb800 0x55dc846a3b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc8474d400 msgr2=0x55dc846a4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc8474d400 0x55dc846a4c00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc8474dc00 msgr2=0x55dc846a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.295+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc8474dc00 0x55dc846a4100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474cc00 msgr2=0x55dc846a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474cc00 0x55dc846a3600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55dc8518a030 tx=0x55dc847fc0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 msgr2=0x55dc846a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1316101019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843bb400 0x55dc846a2b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55dc8523c930 tx=0x55dc844abce0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 reap_dead start 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f51490ed700 1 -- 172.21.15.191:0/1316101019 reap_dead start 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 shutdown_connections 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 >> 172.21.15.191:0/1316101019 conn(0x55dc843bac00 msgr2=0x55dc844a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 shutdown_connections 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1316101019 wait complete. 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8480c800 msgr2=0x55dc848c4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dc8480c800 0x55dc848c4580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc8480c000 msgr2=0x55dc848c4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dc8480c000 0x55dc848c4000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc848af400 msgr2=0x55dc848c5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dc848af400 0x55dc848c5080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc848ae800 msgr2=0x55dc848c4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dc848ae800 0x55dc848c4b00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc848ae000 msgr2=0x55dc846a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc848ae000 0x55dc846a5700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55dc847a94a0 tx=0x55dc847fc360 comp rx=0 tx=0).stop 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 msgr2=0x55dc846a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 --2- 172.21.15.191:0/1735810487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc8480d400 0x55dc846a4680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55dc8481d200 tx=0x55dc847fc4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f51498ee700 1 -- 172.21.15.191:0/1735810487 reap_dead start 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 shutdown_connections 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 >> 172.21.15.191:0/1735810487 conn(0x55dc8480d000 msgr2=0x55dc844a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 shutdown_connections 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.296+0000 7f514b944a80 1 -- 172.21.15.191:0/1735810487 wait complete. 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474c000 msgr2=0x55dc846a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474c000 0x55dc846a3080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55dc8523d080 tx=0x55dc84712b40 comp rx=0 tx=0).stop 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 msgr2=0x55dc846a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55dc8518bec0 tx=0x55dc844ab980 comp rx=0 tx=0).stop 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 shutdown_connections 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dc8474c000 0x55dc846a3080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 --2- 172.21.15.191:0/3871726645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc843ba800 0x55dc846a2580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 >> 172.21.15.191:0/3871726645 conn(0x55dc843ba400 msgr2=0x55dc844a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 shutdown_connections 2024-04-27T04:34:42.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.297+0000 7f514b944a80 1 -- 172.21.15.191:0/3871726645 wait complete. 2024-04-27T04:34:42.303 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T04:34:42.303 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-04-27T04:34:42.304 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-04-27T04:34:42.304 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-04-27T04:34:42.349 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.349 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.348+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.348+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.348+0000 7fea2c48ba80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55c5721fac00 0x55c5724e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.348+0000 7fea2c48ba80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c572525500 con 0x55c5721fa800 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.348+0000 7fea2c48ba80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c57229ca00 con 0x55c5721fac00 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea2a435700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55c5721fa800 0x55c5722e6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55210/0 (socket says 172.21.15.191:55210) 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29c34700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55c5721fac00 0x55c5724e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea2a435700 1 -- 172.21.15.191:0/3298807555 learned_addr learned my addr 172.21.15.191:0/3298807555 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29433700 1 -- 172.21.15.191:0/3298807555 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469063791 0 0) 0x55c572fc8000 con 0x55c5721fa800 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29433700 1 -- 172.21.15.191:0/3298807555 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55c572525680 con 0x55c5721fa800 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29c34700 1 -- 172.21.15.191:0/3298807555 >> v1:172.21.15.138:6789/0 conn(0x55c5721fa800 legacy=0x55c5722e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29c34700 1 -- 172.21.15.191:0/3298807555 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572223d40 con 0x55c5721fac00 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29c34700 1 --2- 172.21.15.191:0/3298807555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fac00 0x55c5724e2580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55c5724dd890 tx=0x55c5722eb980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8cb5b4cda8929e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29433700 1 -- 172.21.15.191:0/3298807555 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b34a0 con 0x55c5721fac00 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29433700 1 -- 172.21.15.191:0/3298807555 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c572525500 con 0x55c5721fac00 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.349+0000 7fea29433700 1 -- 172.21.15.191:0/3298807555 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b3600 con 0x55c5721fac00 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3298807555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fac00 msgr2=0x55c5724e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3298807555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fac00 0x55c5724e2580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55c5724dd890 tx=0x55c5722eb980 comp rx=0 tx=0).stop 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3298807555 shutdown_connections 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3298807555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fac00 0x55c5724e2580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3298807555 >> 172.21.15.191:0/3298807555 conn(0x55c5721fa400 msgr2=0x55c5722e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3298807555 shutdown_connections 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3298807555 wait complete. 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.350+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2c48ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2c48ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57229c500 con 0x55c5721fa800 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2a435700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2a435700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38012/0 (socket says 172.21.15.191:38012) 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2a435700 1 -- 172.21.15.191:0/356369633 learned_addr learned my addr 172.21.15.191:0/356369633 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2a435700 1 -- 172.21.15.191:0/356369633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572223ba0 con 0x55c5721fa800 2024-04-27T04:34:42.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea2a435700 1 --2- 172.21.15.191:0/356369633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55c572fcb770 tx=0x55c5722ebb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97526badeaef910f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea28c32700 1 -- 172.21.15.191:0/356369633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b31e0 con 0x55c5721fa800 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea28c32700 1 -- 172.21.15.191:0/356369633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c572525b00 con 0x55c5721fa800 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.351+0000 7fea28c32700 1 -- 172.21.15.191:0/356369633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b34a0 con 0x55c5721fa800 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- 172.21.15.191:0/356369633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 msgr2=0x55c5724e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/356369633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55c572fcb770 tx=0x55c5722ebb60 comp rx=0 tx=0).stop 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- 172.21.15.191:0/356369633 shutdown_connections 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/356369633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- 172.21.15.191:0/356369633 >> 172.21.15.191:0/356369633 conn(0x55c5721fa400 msgr2=0x55c5722e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- 172.21.15.191:0/356369633 shutdown_connections 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- 172.21.15.191:0/356369633 wait complete. 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2c48ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57229c3c0 con 0x55c5721fa800 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2a435700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2a435700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38022/0 (socket says 172.21.15.191:38022) 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.352+0000 7fea2a435700 1 -- 172.21.15.191:0/3356608966 learned_addr learned my addr 172.21.15.191:0/3356608966 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2a435700 1 -- 172.21.15.191:0/3356608966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572223a00 con 0x55c5721fa800 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2a435700 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55c572fcbe30 tx=0x55c5722eb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2742f700 1 -- 172.21.15.191:0/3356608966 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b3600 con 0x55c5721fa800 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2742f700 1 -- 172.21.15.191:0/3356608966 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c57229bc80 con 0x55c5721fa800 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c572223860 con 0x55c5721fa800 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2742f700 1 -- 172.21.15.191:0/3356608966 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b31e0 con 0x55c5721fa800 2024-04-27T04:34:42.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5722236c0 con 0x55c5721fa800 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.353+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2742f700 1 -- 172.21.15.191:0/3356608966 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c572f9cb00 con 0x55c5721fa800 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2c48ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2c48ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c572236640 con 0x55c5721fb400 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2742f700 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586000 0x55c5724e3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2ac36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2742f700 1 -- 172.21.15.191:0/3356608966 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c572275180 con 0x55c5721fa800 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2ac36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38026/0 (socket says 172.21.15.191:38026) 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2ac36700 1 -- 172.21.15.191:0/4086849292 learned_addr learned my addr 172.21.15.191:0/4086849292 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2a435700 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586000 0x55c5724e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2ac36700 1 -- 172.21.15.191:0/4086849292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572222000 con 0x55c5721fb400 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2a435700 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586000 0x55c5724e3080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55c573088f00 tx=0x55c572544240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea2ac36700 1 --2- 172.21.15.191:0/4086849292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55c572fcbe90 tx=0x55c572544de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc23b286a6b3155 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea24c2a700 1 -- 172.21.15.191:0/4086849292 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b3600 con 0x55c5721fb400 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea24c2a700 1 -- 172.21.15.191:0/4086849292 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c572524480 con 0x55c5721fb400 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.354+0000 7fea24c2a700 1 -- 172.21.15.191:0/4086849292 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b31e0 con 0x55c5721fb400 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- 172.21.15.191:0/4086849292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 msgr2=0x55c5724e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/4086849292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55c572fcbe90 tx=0x55c572544de0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- 172.21.15.191:0/4086849292 shutdown_connections 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/4086849292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- 172.21.15.191:0/4086849292 >> 172.21.15.191:0/4086849292 conn(0x55c5721fac00 msgr2=0x55c5722e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- 172.21.15.191:0/4086849292 shutdown_connections 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- 172.21.15.191:0/4086849292 wait complete. 2024-04-27T04:34:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2c48ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5722368c0 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.355+0000 7fea2ac36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea2ac36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38040/0 (socket says 172.21.15.191:38040) 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 learned_addr learned my addr 172.21.15.191:0/339903896 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572222ea0 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea2ac36700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55c573088450 tx=0x55c5722ebf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea23427700 1 -- 172.21.15.191:0/339903896 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b3a20 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea23427700 1 -- 172.21.15.191:0/339903896 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c572524180 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c572222820 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea23427700 1 -- 172.21.15.191:0/339903896 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b3600 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.356+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5722229c0 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea23427700 1 -- 172.21.15.191:0/339903896 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c572f9cdc0 con 0x55c5721fb400 2024-04-27T04:34:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea23427700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586c00 0x55c5724e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea23427700 1 -- 172.21.15.191:0/339903896 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c572275340 con 0x55c5721fb400 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea29c34700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586c00 0x55c5724e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c5721fb800 0x55c5724e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5721fbc00 con 0x55c5721fb800 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.357+0000 7fea2ac36700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c5721fb800 0x55c5724e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea29c34700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586c00 0x55c5724e3600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55c5724dd890 tx=0x55c5722eb800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2ac36700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c5721fb800 0x55c5724e3b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 realm 2024-04-27T04:34:42.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587000 con 0x55c5721fb800 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587400 con 0x55c5721fb800 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.358+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587800 con 0x55c5721fb800 2024-04-27T04:34:42.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c572587c00 0x55c5724e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587000 con 0x55c572587c00 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2a435700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c572587c00 0x55c5724e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2a435700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c572587c00 0x55c5724e4100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 20 searching for the correct realm 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.359+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587400 con 0x55c572587c00 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea2a435700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c572587400 0x55c5724e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587000 con 0x55c572587400 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea29c34700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c572587400 0x55c5724e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea29c34700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c572587400 0x55c5724e4c00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea29c34700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c572587000 0x55c5724e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.360+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.361+0000 7fea2ac36700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c572587000 0x55c5724e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.361+0000 7fea2ac36700 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c572587000 0x55c5724e5180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.361+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.361+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.361+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.361+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587400 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.362+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587400 2024-04-27T04:34:42.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.363+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.364+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.365+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587400 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.366+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587400 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587c00 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.367+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587400 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:42.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5721fbc00 con 0x55c5721fb800 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c400 con 0x55c5721fb800 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.368+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c800 con 0x55c572587c00 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:42.386 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 zone default found 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 4 Realm: () 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 zonegroup default 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264cc00 con 0x55c572587000 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d000 con 0x55c572587c00 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d400 con 0x55c572587c00 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 started sync module instance, tier type = 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.369+0000 7fea2c48ba80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.370+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c572587400 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.370+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.370+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94306514606848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c800 con 0x55c572587400 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.371+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d000 con 0x55c5721fb800 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.372+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94306514606848] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.372+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.372+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94306514608000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d400 con 0x55c5721fb800 2024-04-27T04:34:42.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.372+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c400 con 0x55c572587c00 2024-04-27T04:34:42.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.373+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94306514608000] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.373+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.373+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94306514609152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264cc00 con 0x55c572587c00 2024-04-27T04:34:42.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.373+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5721fbc00 con 0x55c572587400 2024-04-27T04:34:42.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.374+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94306514609152] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.374+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.374+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94306514610304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572587800 con 0x55c572587400 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.374+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c5721fb800 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.375+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.375+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94306514611456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c5721fb800 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.375+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c800 con 0x55c572587000 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.376+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94306514610304] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.376+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.376+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94306514612608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264dc00 con 0x55c572587000 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.376+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d000 con 0x55c572587000 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.376+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94306514611456] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.377+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.377+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94306514613760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d400 con 0x55c572587000 2024-04-27T04:34:42.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.377+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c000 con 0x55c572587000 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.377+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94306514612608] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94306514613760] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94306514614912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c572587000 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=0 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=1 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=2 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=3 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=4 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=5 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.378+0000 7fea2c48ba80 20 add_watcher() i=6 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.379+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94306514614912] v31'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.379+0000 7fea2c48ba80 20 add_watcher() i=7 2024-04-27T04:34:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.379+0000 7fea2c48ba80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.379+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c800 con 0x55c5721fb800 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.379+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.379+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94306516172800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264dc00 con 0x55c5721fb800 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.380+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94306516172800] v31'37 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.380+0000 7fea11403700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.380+0000 7fea2c48ba80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.380+0000 7fea2c48ba80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea2c48ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea2c48ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57229c3c0 con 0x55c57264d400 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea29c34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea29c34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38046/0 (socket says 172.21.15.191:38046) 2024-04-27T04:34:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea29c34700 1 -- 172.21.15.191:0/2210958711 learned_addr learned my addr 172.21.15.191:0/2210958711 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.381+0000 7fea29c34700 1 -- 172.21.15.191:0/2210958711 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572222b60 con 0x55c57264d400 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea29c34700 1 --2- 172.21.15.191:0/2210958711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55c57259b440 tx=0x55c5722eb9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363c93dcad2d8fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea10c02700 1 -- 172.21.15.191:0/2210958711 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5722b3760 con 0x55c57264d400 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea10c02700 1 -- 172.21.15.191:0/2210958711 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c572662000 con 0x55c57264d400 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea10c02700 1 -- 172.21.15.191:0/2210958711 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c57266c000 con 0x55c57264d400 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 -- 172.21.15.191:0/2210958711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 msgr2=0x55c5724e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/2210958711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55c57259b440 tx=0x55c5722eb9e0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 -- 172.21.15.191:0/2210958711 shutdown_connections 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/2210958711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 -- 172.21.15.191:0/2210958711 >> 172.21.15.191:0/2210958711 conn(0x55c57264d000 msgr2=0x55c5722e8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 -- 172.21.15.191:0/2210958711 shutdown_connections 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.382+0000 7fea2c48ba80 1 -- 172.21.15.191:0/2210958711 wait complete. 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea2c48ba80 1 Processor -- start 2024-04-27T04:34:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea2c48ba80 1 -- start start 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea2c48ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea2c48ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57229c500 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea29c34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea29c34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38054/0 (socket says 172.21.15.191:38054) 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 learned_addr learned my addr 172.21.15.191:0/3546437011 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c572222d00 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.383+0000 7fea29c34700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55c572661260 tx=0x55c5722eb8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.384+0000 7fea0f3ff700 1 -- 172.21.15.191:0/3546437011 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c57266c160 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.384+0000 7fea0f3ff700 1 -- 172.21.15.191:0/3546437011 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c572662180 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.384+0000 7fea0f3ff700 1 -- 172.21.15.191:0/3546437011 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c57266c2c0 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.384+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c572222340 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.384+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c572223d40 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.384+0000 7fea0f3ff700 1 -- 172.21.15.191:0/3546437011 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c572f9db80 con 0x55c57264d400 2024-04-27T04:34:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea0f3ff700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c5726e6000 0x55c5724e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea0f3ff700 1 -- 172.21.15.191:0/3546437011 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c572275500 con 0x55c57264d400 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea2ac36700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c5726e6000 0x55c5724e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c57264c000 0x55c572702000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c57264c000 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea2a435700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c57264c000 0x55c572702000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea2ac36700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c5726e6000 0x55c5724e5700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55c5725eb170 tx=0x55c572673200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.385+0000 7fea2a435700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c57264c000 0x55c572702000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.386+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'41 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c57264c000 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.386+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c57264c800 0x55c572702580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.386+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e6400 con 0x55c57264c800 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.386+0000 7fea29c34700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c57264c800 0x55c572702580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.387+0000 7fea29c34700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c57264c800 0x55c572702580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.388+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'17 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.388+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c5726e6800 0x55c572702b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.388+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e6c00 con 0x55c5726e6800 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.388+0000 7fea2ac36700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c5726e6800 0x55c572702b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.388+0000 7fea2ac36700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c5726e6800 0x55c572702b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.389+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.389+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7000 con 0x55c57264c000 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.390+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'38 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c57264c000 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.390+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c5726e7400 0x55c572703080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.390+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7800 con 0x55c5726e7400 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.390+0000 7fea2a435700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c5726e7400 0x55c572703080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.390+0000 7fea2a435700 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c5726e7400 0x55c572703080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.391+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'29 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.391+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7c00 con 0x55c5726e6800 2024-04-27T04:34:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.392+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'27 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.392+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7000 con 0x55c57264c000 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.393+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c57264c000 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.393+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c400 con 0x55c57264c800 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.394+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'23 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.394+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264cc00 con 0x55c5726e7400 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.395+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.395+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c5726e6800 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.396+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.396+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734000 con 0x55c5726e7400 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.397+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'30 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.397+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734400 con 0x55c57264c800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.398+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.398+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734800 con 0x55c5726e7400 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.399+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.399+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734c00 con 0x55c57264c800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.400+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'33 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.400+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735000 con 0x55c57264c000 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.401+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'39 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c57264c000 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.401+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735400 con 0x55c5726e6800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.402+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.402+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735800 con 0x55c5726e7400 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.403+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'31 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.403+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735c00 con 0x55c5726e6800 2024-04-27T04:34:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.404+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'23 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.404+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734000 con 0x55c57264c000 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.405+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c57264c000 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.405+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734800 con 0x55c57264c000 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.405+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'40 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c57264c000 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.406+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734400 con 0x55c5726e6800 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.406+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'24 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.406+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264cc00 con 0x55c5726e7400 2024-04-27T04:34:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.408+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.408+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c400 con 0x55c5726e6800 2024-04-27T04:34:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.409+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.409+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7800 con 0x55c5726e6800 2024-04-27T04:34:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.410+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'29 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.410+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7000 con 0x55c5726e6800 2024-04-27T04:34:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.411+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.411+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e6400 con 0x55c5726e6800 2024-04-27T04:34:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.412+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'28 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.412+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5721fbc00 con 0x55c57264c800 2024-04-27T04:34:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.413+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.413+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735400 con 0x55c5726e6800 2024-04-27T04:34:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.414+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'23 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.414+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735c00 con 0x55c57264c800 2024-04-27T04:34:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.415+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'24 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.415+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734400 con 0x55c5726e6800 2024-04-27T04:34:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.416+0000 7fea2ac36700 1 -- 172.21.15.191:0/3546437011 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'25 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5726e6800 2024-04-27T04:34:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.416+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264dc00 con 0x55c57264c800 2024-04-27T04:34:42.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.417+0000 7fea29c34700 1 -- 172.21.15.191:0/3546437011 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'34 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c57264c800 2024-04-27T04:34:42.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.417+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c5726e7400 2024-04-27T04:34:42.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.418+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c5726e7400 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea0cbfa700 20 reqs_thread_entry: start 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea0abf6700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:12 2024 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea0a3d4700 20 reqs_thread_entry: start 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5726e7c00 con 0x55c572587000 2024-04-27T04:34:42.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-27T04:34:42.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.419+0000 7fea2c48ba80 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T04:34:42.422 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T04:34:42.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.420+0000 7fea11403700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.420+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94306516172800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264cc00 con 0x55c5721fb800 2024-04-27T04:34:42.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.421+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94306516172800] v31'41 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.421+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94306514606848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c572587400 2024-04-27T04:34:42.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.422+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94306514606848] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.422+0000 7fea2c48ba80 20 remove_watcher() i=0 2024-04-27T04:34:42.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.422+0000 7fea2c48ba80 2 removed watcher, disabling cache 2024-04-27T04:34:42.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.422+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94306514608000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735000 con 0x55c5721fb800 2024-04-27T04:34:42.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.423+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94306514608000] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.423+0000 7fea2c48ba80 20 remove_watcher() i=1 2024-04-27T04:34:42.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.423+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94306514609152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572735800 con 0x55c572587c00 2024-04-27T04:34:42.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.424+0000 7fea2a435700 1 -- 172.21.15.191:0/339903896 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94306514609152] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641200 con 0x55c572587c00 2024-04-27T04:34:42.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.424+0000 7fea2c48ba80 20 remove_watcher() i=2 2024-04-27T04:34:42.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.424+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94306514610304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734000 con 0x55c572587400 2024-04-27T04:34:42.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.425+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94306514610304] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572641440 con 0x55c572587400 2024-04-27T04:34:42.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.425+0000 7fea2c48ba80 20 remove_watcher() i=3 2024-04-27T04:34:42.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.425+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94306514611456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c572734800 con 0x55c5721fb800 2024-04-27T04:34:42.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.426+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94306514611456] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c5721fb800 2024-04-27T04:34:42.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.426+0000 7fea2c48ba80 20 remove_watcher() i=4 2024-04-27T04:34:42.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.426+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94306514612608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264dc00 con 0x55c572587000 2024-04-27T04:34:42.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.427+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94306514612608] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.427+0000 7fea2c48ba80 20 remove_watcher() i=5 2024-04-27T04:34:42.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.427+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94306514613760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264d800 con 0x55c572587000 2024-04-27T04:34:42.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.428+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94306514613760] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.428+0000 7fea2c48ba80 20 remove_watcher() i=6 2024-04-27T04:34:42.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.428+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94306514614912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c57264c400 con 0x55c572587000 2024-04-27T04:34:42.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2ac36700 1 -- 172.21.15.191:0/339903896 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94306514614912] v31'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c572640b40 con 0x55c572587000 2024-04-27T04:34:42.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 20 remove_watcher() i=7 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c572587000 msgr2=0x55c5724e5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c572587000 0x55c5724e5180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c5721fb800 msgr2=0x55c5724e3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c5721fb800 0x55c5724e3b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c572587400 msgr2=0x55c5724e4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c572587400 0x55c5724e4c00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c572587c00 msgr2=0x55c5724e4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c572587c00 0x55c5724e4100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586c00 msgr2=0x55c5724e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586c00 0x55c5724e3600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55c5724dd890 tx=0x55c5722eb800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 msgr2=0x55c5724e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/339903896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fb400 0x55c5724e2b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55c573088450 tx=0x55c5722ebf20 comp rx=0 tx=0).stop 2024-04-27T04:34:42.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 reap_dead start 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea29c34700 1 -- 172.21.15.191:0/339903896 reap_dead start 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 shutdown_connections 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 >> 172.21.15.191:0/339903896 conn(0x55c5721fac00 msgr2=0x55c5722e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 shutdown_connections 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.429+0000 7fea2c48ba80 1 -- 172.21.15.191:0/339903896 wait complete. 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c57264c800 msgr2=0x55c572702580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c57264c800 0x55c572702580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c57264c000 msgr2=0x55c572702000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c57264c000 0x55c572702000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c5726e7400 msgr2=0x55c572703080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c5726e7400 0x55c572703080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c5726e6800 msgr2=0x55c572702b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c5726e6800 0x55c572702b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c5726e6000 msgr2=0x55c5724e5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c5726e6000 0x55c5724e5700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55c5725eb170 tx=0x55c572673200 comp rx=0 tx=0).stop 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 msgr2=0x55c5724e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2a435700 1 -- 172.21.15.191:0/3546437011 reap_dead start 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55c572661260 tx=0x55c5722eb8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 shutdown_connections 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3546437011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57264d400 0x55c5724e4680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 >> 172.21.15.191:0/3546437011 conn(0x55c57264d000 msgr2=0x55c5722e8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 shutdown_connections 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3546437011 wait complete. 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586000 msgr2=0x55c5724e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586000 0x55c5724e3080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55c573088f00 tx=0x55c572544240 comp rx=0 tx=0).stop 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 msgr2=0x55c5724e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55c572fcbe30 tx=0x55c5722eb920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 shutdown_connections 2024-04-27T04:34:42.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c572586000 0x55c5724e3080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 --2- 172.21.15.191:0/3356608966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5721fa800 0x55c5724e2580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 >> 172.21.15.191:0/3356608966 conn(0x55c5721fa400 msgr2=0x55c5722e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 shutdown_connections 2024-04-27T04:34:42.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.430+0000 7fea2c48ba80 1 -- 172.21.15.191:0/3356608966 wait complete. 2024-04-27T04:34:42.436 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T04:34:42.437 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T04:34:42.437 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T04:34:42.437 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T04:34:42.482 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.483 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f77181b1a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564162ff0c00 0x5641632d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f77181b1a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56416331b500 con 0x564162ff0800 2024-04-27T04:34:42.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f77181b1a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x564163092a00 con 0x564162ff0c00 2024-04-27T04:34:42.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f771615b700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x564162ff0800 0x5641630dc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55222/0 (socket says 172.21.15.191:55222) 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f771615b700 1 -- 172.21.15.191:0/1286658458 learned_addr learned my addr 172.21.15.191:0/1286658458 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f771595a700 1 --2- 172.21.15.191:0/1286658458 >> v2:172.21.15.138:3300/0 conn(0x564162ff0c00 0x5641632d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f7715159700 1 -- 172.21.15.191:0/1286658458 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013223246 0 0) 0x564163dbe000 con 0x564162ff0800 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.482+0000 7f7715159700 1 -- 172.21.15.191:0/1286658458 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x56416331b680 con 0x564162ff0800 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f771595a700 1 -- 172.21.15.191:0/1286658458 >> v1:172.21.15.138:6789/0 conn(0x564162ff0800 legacy=0x5641630dc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f771595a700 1 -- 172.21.15.191:0/1286658458 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163019d40 con 0x564162ff0c00 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f771595a700 1 --2- 172.21.15.191:0/1286658458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0c00 0x5641632d8580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5641632d3890 tx=0x5641630e1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e74187dd61947f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f7715159700 1 -- 172.21.15.191:0/1286658458 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a94a0 con 0x564162ff0c00 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f7715159700 1 -- 172.21.15.191:0/1286658458 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56416331b680 con 0x564162ff0c00 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f7715159700 1 -- 172.21.15.191:0/1286658458 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a9600 con 0x564162ff0c00 2024-04-27T04:34:42.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 -- 172.21.15.191:0/1286658458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0c00 msgr2=0x5641632d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 --2- 172.21.15.191:0/1286658458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0c00 0x5641632d8580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5641632d3890 tx=0x5641630e1980 comp rx=0 tx=0).stop 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 -- 172.21.15.191:0/1286658458 shutdown_connections 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 --2- 172.21.15.191:0/1286658458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0c00 0x5641632d8580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 -- 172.21.15.191:0/1286658458 >> 172.21.15.191:0/1286658458 conn(0x564162ff0400 msgr2=0x5641630dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 -- 172.21.15.191:0/1286658458 shutdown_connections 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.483+0000 7f77181b1a80 1 -- 172.21.15.191:0/1286658458 wait complete. 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f77181b1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f77181b1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564163092500 con 0x564162ff0800 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f771615b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f771615b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38072/0 (socket says 172.21.15.191:38072) 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.484+0000 7f771615b700 1 -- 172.21.15.191:0/3055477041 learned_addr learned my addr 172.21.15.191:0/3055477041 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f771615b700 1 -- 172.21.15.191:0/3055477041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163019ba0 con 0x564162ff0800 2024-04-27T04:34:42.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f771615b700 1 --2- 172.21.15.191:0/3055477041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x564163dc1770 tx=0x5641630e1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec128be73af76a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f7714958700 1 -- 172.21.15.191:0/3055477041 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a94a0 con 0x564162ff0800 2024-04-27T04:34:42.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f7714958700 1 -- 172.21.15.191:0/3055477041 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56416331bb00 con 0x564162ff0800 2024-04-27T04:34:42.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f7714958700 1 -- 172.21.15.191:0/3055477041 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a9600 con 0x564162ff0800 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 -- 172.21.15.191:0/3055477041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 msgr2=0x5641632d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3055477041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x564163dc1770 tx=0x5641630e1b60 comp rx=0 tx=0).stop 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 -- 172.21.15.191:0/3055477041 shutdown_connections 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3055477041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 -- 172.21.15.191:0/3055477041 >> 172.21.15.191:0/3055477041 conn(0x564162ff0400 msgr2=0x5641630dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 -- 172.21.15.191:0/3055477041 shutdown_connections 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.485+0000 7f77181b1a80 1 -- 172.21.15.191:0/3055477041 wait complete. 2024-04-27T04:34:42.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f77181b1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f77181b1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641630923c0 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f771615b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f771615b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38088/0 (socket says 172.21.15.191:38088) 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f771615b700 1 -- 172.21.15.191:0/3249402841 learned_addr learned my addr 172.21.15.191:0/3249402841 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f771615b700 1 -- 172.21.15.191:0/3249402841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163019a00 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.486+0000 7f771615b700 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x564163dc1e30 tx=0x5641630e1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f7713155700 1 -- 172.21.15.191:0/3249402841 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a98c0 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f7713155700 1 -- 172.21.15.191:0/3249402841 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564163091c80 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564163019860 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f7713155700 1 -- 172.21.15.191:0/3249402841 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a94a0 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641630196c0 con 0x564162ff0800 2024-04-27T04:34:42.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f77181b1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f77181b1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416302c640 con 0x564162ff1400 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.487+0000 7f7713155700 1 -- 172.21.15.191:0/3249402841 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564163d92b00 con 0x564162ff0800 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f7713155700 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337c000 0x5641632d9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771695c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f7713155700 1 -- 172.21.15.191:0/3249402841 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56416306b180 con 0x564162ff0800 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771695c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38098/0 (socket says 172.21.15.191:38098) 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771695c700 1 -- 172.21.15.191:0/651982007 learned_addr learned my addr 172.21.15.191:0/651982007 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771615b700 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337c000 0x5641632d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771695c700 1 -- 172.21.15.191:0/651982007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163018000 con 0x564162ff1400 2024-04-27T04:34:42.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771615b700 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337c000 0x5641632d9080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x564163e7ef30 tx=0x56416333c240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f771695c700 1 --2- 172.21.15.191:0/651982007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x564163dc1ef0 tx=0x56416333cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b79ab79341008e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f7710950700 1 -- 172.21.15.191:0/651982007 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a98c0 con 0x564162ff1400 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f7710950700 1 -- 172.21.15.191:0/651982007 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56416331a480 con 0x564162ff1400 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.488+0000 7f7710950700 1 -- 172.21.15.191:0/651982007 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a94a0 con 0x564162ff1400 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- 172.21.15.191:0/651982007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 msgr2=0x5641632d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 --2- 172.21.15.191:0/651982007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x564163dc1ef0 tx=0x56416333cde0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- 172.21.15.191:0/651982007 shutdown_connections 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 --2- 172.21.15.191:0/651982007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- 172.21.15.191:0/651982007 >> 172.21.15.191:0/651982007 conn(0x564162ff0c00 msgr2=0x5641630dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- 172.21.15.191:0/651982007 shutdown_connections 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- 172.21.15.191:0/651982007 wait complete. 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f77181b1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416302c8c0 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f771695c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f771695c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38112/0 (socket says 172.21.15.191:38112) 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.489+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 learned_addr learned my addr 172.21.15.191:0/3670036861 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163018ea0 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f771695c700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x564163e7e420 tx=0x5641630e1f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f770f14d700 1 -- 172.21.15.191:0/3670036861 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a9b80 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f770f14d700 1 -- 172.21.15.191:0/3670036861 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56416331a180 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564163018820 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f770f14d700 1 -- 172.21.15.191:0/3670036861 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a98c0 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.490+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641630189c0 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f770f14d700 1 -- 172.21.15.191:0/3670036861 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564163d92dc0 con 0x564162ff1400 2024-04-27T04:34:42.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f770f14d700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337cc00 0x5641632d9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f770f14d700 1 -- 172.21.15.191:0/3670036861 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56416306b340 con 0x564162ff1400 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f771595a700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337cc00 0x5641632d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f771595a700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337cc00 0x5641632d9600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5641632d3890 tx=0x5641630e1800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564162ff1800 0x5641632d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564162ff1c00 con 0x564162ff1800 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.491+0000 7f771695c700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564162ff1800 0x5641632d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f771695c700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564162ff1800 0x5641632d9b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 realm 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d000 con 0x564162ff1800 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d400 con 0x564162ff1800 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.492+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d800 con 0x564162ff1800 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56416337dc00 0x5641632da100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d000 con 0x56416337dc00 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f771615b700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56416337dc00 0x5641632da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f771615b700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56416337dc00 0x5641632da100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 20 searching for the correct realm 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.493+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d400 con 0x56416337dc00 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771615b700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56416337d400 0x5641632dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d000 con 0x56416337d400 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771595a700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56416337d400 0x5641632dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771595a700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56416337d400 0x5641632dac00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771595a700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56416337d000 0x5641632db180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.494+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.495+0000 7f771695c700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56416337d000 0x5641632db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.495+0000 7f771695c700 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56416337d000 0x5641632db180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.495+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.495+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.495+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.495+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d400 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.496+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d400 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.497+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.498+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.499+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d400 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.500+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.501+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d400 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.501+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.501+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337dc00 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.501+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.501+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d400 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:42.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564162ff1c00 con 0x564162ff1800 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442400 con 0x564162ff1800 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.502+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442800 con 0x56416337dc00 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 zone default found 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 4 Realm: () 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 zonegroup default 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442c00 con 0x56416337d000 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443000 con 0x56416337dc00 2024-04-27T04:34:42.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443400 con 0x56416337dc00 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.503+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.504+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.504+0000 7f77181b1a80 20 started sync module instance, tier type = 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.504+0000 7f77181b1a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.504+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x56416337d400 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.505+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.505+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94838836755200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442800 con 0x56416337d400 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.505+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443000 con 0x564162ff1800 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.506+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94838836755200] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.506+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.506+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94838836756352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443400 con 0x564162ff1800 2024-04-27T04:34:42.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.506+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442400 con 0x56416337dc00 2024-04-27T04:34:42.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.507+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94838836756352] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.507+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.507+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94838836757504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442c00 con 0x56416337dc00 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.507+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564162ff1c00 con 0x56416337d400 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.508+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.508+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94838836757504] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.508+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94838836758656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416337d800 con 0x56416337d400 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.508+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x564162ff1800 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.509+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.509+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94838836759808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x564162ff1800 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.509+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442800 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.509+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94838836758656] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.510+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94838836759808] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.510+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.510+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94838836760960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443c00 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.510+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443000 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.511+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94838836760960] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.511+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.511+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94838836762112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443400 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.511+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442000 con 0x56416337d000 2024-04-27T04:34:42.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94838836762112] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94838836763264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x56416337d000 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=0 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=1 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=2 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=3 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=4 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=5 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.512+0000 7f77181b1a80 20 add_watcher() i=6 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.513+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94838836763264] v31'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.513+0000 7f77181b1a80 20 add_watcher() i=7 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.513+0000 7f77181b1a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.513+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442800 con 0x564162ff1800 2024-04-27T04:34:42.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.513+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.513+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94838838321152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443c00 con 0x564162ff1800 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94838838321152] v31'42 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f76fd129700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f77181b1a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f77181b1a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f77181b1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f77181b1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641630923c0 con 0x564163443400 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f771595a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f771595a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38128/0 (socket says 172.21.15.191:38128) 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.516+0000 7f771595a700 1 -- 172.21.15.191:0/1147435131 learned_addr learned my addr 172.21.15.191:0/1147435131 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f771595a700 1 -- 172.21.15.191:0/1147435131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163018b60 con 0x564163443400 2024-04-27T04:34:42.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f771595a700 1 --2- 172.21.15.191:0/1147435131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5641633e5800 tx=0x5641630e19e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c2e0f6b175dcd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f76fc928700 1 -- 172.21.15.191:0/1147435131 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641630a9760 con 0x564163443400 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f76fc928700 1 -- 172.21.15.191:0/1147435131 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564163458000 con 0x564163443400 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f76fc928700 1 -- 172.21.15.191:0/1147435131 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564163468000 con 0x564163443400 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f77181b1a80 1 -- 172.21.15.191:0/1147435131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 msgr2=0x5641632da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.517+0000 7f77181b1a80 1 --2- 172.21.15.191:0/1147435131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5641633e5800 tx=0x5641630e19e0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 -- 172.21.15.191:0/1147435131 shutdown_connections 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 --2- 172.21.15.191:0/1147435131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 -- 172.21.15.191:0/1147435131 >> 172.21.15.191:0/1147435131 conn(0x564163443000 msgr2=0x5641630de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 -- 172.21.15.191:0/1147435131 shutdown_connections 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 -- 172.21.15.191:0/1147435131 wait complete. 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 Processor -- start 2024-04-27T04:34:42.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 -- start start 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f77181b1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564163092500 con 0x564163443400 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f771595a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f771595a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38140/0 (socket says 172.21.15.191:38140) 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 learned_addr learned my addr 172.21.15.191:0/742910200 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.518+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564163018d00 con 0x564163443400 2024-04-27T04:34:42.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f771595a700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x564163457260 tx=0x5641630e18c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f76fb125700 1 -- 172.21.15.191:0/742910200 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564163468160 con 0x564163443400 2024-04-27T04:34:42.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f76fb125700 1 -- 172.21.15.191:0/742910200 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564163458180 con 0x564163443400 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564163018340 con 0x564163443400 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f76fb125700 1 -- 172.21.15.191:0/742910200 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641634682c0 con 0x564163443400 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564163019d40 con 0x564163443400 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.519+0000 7f76fb125700 1 -- 172.21.15.191:0/742910200 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564163d93b80 con 0x564163443400 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f76fb125700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641634e2000 0x5641632db700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f76fb125700 1 -- 172.21.15.191:0/742910200 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56416306b500 con 0x564163443400 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f771695c700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641634e2000 0x5641632db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564163442000 0x5641634f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x564163442000 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f771615b700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564163442000 0x5641634f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f771695c700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641634e2000 0x5641632db700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5641633e5320 tx=0x564163467320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.520+0000 7f771615b700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564163442000 0x5641634f8000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.521+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'42 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343afc0 con 0x564163442000 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.521+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564163442800 0x5641634f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.521+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e2400 con 0x564163442800 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.521+0000 7f771595a700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564163442800 0x5641634f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.522+0000 7f771595a700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564163442800 0x5641634f8580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.522+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'18 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.523+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641634e2800 0x5641634f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.523+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e2c00 con 0x5641634e2800 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.523+0000 7f771695c700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641634e2800 0x5641634f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.523+0000 7f771695c700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641634e2800 0x5641634f8b00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.525+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.525+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3000 con 0x564163442000 2024-04-27T04:34:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.528+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'43 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343afc0 con 0x564163442000 2024-04-27T04:34:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.528+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641634e3400 0x5641634f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.528+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3800 con 0x5641634e3400 2024-04-27T04:34:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.528+0000 7f771615b700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641634e3400 0x5641634f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.529+0000 7f771615b700 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641634e3400 0x5641634f9080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.531+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'32 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.531+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3c00 con 0x5641634e2800 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.533+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.533+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3000 con 0x564163442000 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.534+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343afc0 con 0x564163442000 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.535+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442400 con 0x564163442800 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.536+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'25 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.536+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442c00 con 0x5641634e3400 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.537+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.537+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x5641634e2800 2024-04-27T04:34:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.540+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.540+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a000 con 0x5641634e3400 2024-04-27T04:34:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.542+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'33 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.542+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a400 con 0x564163442800 2024-04-27T04:34:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.543+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.543+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a800 con 0x5641634e3400 2024-04-27T04:34:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.545+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.545+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352ac00 con 0x564163442800 2024-04-27T04:34:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.546+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'35 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.546+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352b000 con 0x564163442000 2024-04-27T04:34:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.547+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'44 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x564163442000 2024-04-27T04:34:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.547+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352b400 con 0x5641634e2800 2024-04-27T04:34:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.548+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'26 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.548+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352b800 con 0x5641634e3400 2024-04-27T04:34:42.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.549+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'34 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.549+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352bc00 con 0x5641634e2800 2024-04-27T04:34:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.550+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'24 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.550+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a000 con 0x564163442000 2024-04-27T04:34:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.551+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x564163442000 2024-04-27T04:34:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.552+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a800 con 0x564163442000 2024-04-27T04:34:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.554+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'45 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x564163442000 2024-04-27T04:34:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.554+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a400 con 0x5641634e2800 2024-04-27T04:34:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.555+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'27 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.555+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442c00 con 0x5641634e3400 2024-04-27T04:34:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.557+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.557+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442400 con 0x5641634e2800 2024-04-27T04:34:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.560+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.560+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3800 con 0x5641634e2800 2024-04-27T04:34:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.561+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'30 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.561+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3000 con 0x5641634e2800 2024-04-27T04:34:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.561+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.562+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e2400 con 0x5641634e2800 2024-04-27T04:34:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.562+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'30 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.562+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564162ff1c00 con 0x564163442800 2024-04-27T04:34:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.564+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.564+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352b400 con 0x5641634e2800 2024-04-27T04:34:42.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.564+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'24 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.565+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352bc00 con 0x564163442800 2024-04-27T04:34:42.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.565+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'26 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.565+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a400 con 0x5641634e2800 2024-04-27T04:34:42.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.566+0000 7f771695c700 1 -- 172.21.15.191:0/742910200 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'28 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343a900 con 0x5641634e2800 2024-04-27T04:34:42.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.566+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443c00 con 0x564163442800 2024-04-27T04:34:42.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.567+0000 7f771595a700 1 -- 172.21.15.191:0/742910200 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'36 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343b200 con 0x564163442800 2024-04-27T04:34:42.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.567+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x5641634e3400 2024-04-27T04:34:42.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.568+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56416343afc0 con 0x5641634e3400 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.569+0000 7f76f8920700 20 reqs_thread_entry: start 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.569+0000 7f77181b1a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.569+0000 7f77181b1a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.569+0000 7f76f691c700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:13 2024 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f76f60fa700 20 reqs_thread_entry: start 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f77181b1a80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f77181b1a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5641634e3c00 con 0x56416337d000 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f77181b1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f77181b1a80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T04:34:42.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f77181b1a80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:42.573 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T04:34:42.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.570+0000 7f76fd129700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.571+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94838838321152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442c00 con 0x564162ff1800 2024-04-27T04:34:42.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.572+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94838838321152] v31'46 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.572+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94838836755200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x56416337d400 2024-04-27T04:34:42.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.572+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94838836755200] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.573+0000 7f77181b1a80 20 remove_watcher() i=0 2024-04-27T04:34:42.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.573+0000 7f77181b1a80 2 removed watcher, disabling cache 2024-04-27T04:34:42.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.573+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94838836756352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352b000 con 0x564162ff1800 2024-04-27T04:34:42.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.574+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94838836756352] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.574+0000 7f77181b1a80 20 remove_watcher() i=1 2024-04-27T04:34:42.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.574+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94838836757504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352b800 con 0x56416337dc00 2024-04-27T04:34:42.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.575+0000 7f771615b700 1 -- 172.21.15.191:0/3670036861 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94838836757504] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343afc0 con 0x56416337dc00 2024-04-27T04:34:42.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.575+0000 7f77181b1a80 20 remove_watcher() i=2 2024-04-27T04:34:42.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.575+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94838836758656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a000 con 0x56416337d400 2024-04-27T04:34:42.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.576+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94838836758656] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343b200 con 0x56416337d400 2024-04-27T04:34:42.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.576+0000 7f77181b1a80 20 remove_watcher() i=3 2024-04-27T04:34:42.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.576+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94838836759808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56416352a800 con 0x564162ff1800 2024-04-27T04:34:42.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.579+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94838836759808] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x564162ff1800 2024-04-27T04:34:42.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.579+0000 7f77181b1a80 20 remove_watcher() i=4 2024-04-27T04:34:42.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.579+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94838836760960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443c00 con 0x56416337d000 2024-04-27T04:34:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.580+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94838836760960] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.580+0000 7f77181b1a80 20 remove_watcher() i=5 2024-04-27T04:34:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.580+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94838836762112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163443800 con 0x56416337d000 2024-04-27T04:34:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.581+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94838836762112] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.581+0000 7f77181b1a80 20 remove_watcher() i=6 2024-04-27T04:34:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.581+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94838836763264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564163442400 con 0x56416337d000 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f771695c700 1 -- 172.21.15.191:0/3670036861 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94838836763264] v31'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56416343a900 con 0x56416337d000 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 20 remove_watcher() i=7 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56416337d000 msgr2=0x5641632db180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56416337d000 0x5641632db180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564162ff1800 msgr2=0x5641632d9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564162ff1800 0x5641632d9b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56416337d400 msgr2=0x5641632dac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56416337d400 0x5641632dac00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56416337dc00 msgr2=0x5641632da100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56416337dc00 0x5641632da100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337cc00 msgr2=0x5641632d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337cc00 0x5641632d9600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5641632d3890 tx=0x5641630e1800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 msgr2=0x5641632d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3670036861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff1400 0x5641632d8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x564163e7e420 tx=0x5641630e1f20 comp rx=0 tx=0).stop 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 reap_dead start 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f771595a700 1 -- 172.21.15.191:0/3670036861 reap_dead start 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 shutdown_connections 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.583+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 >> 172.21.15.191:0/3670036861 conn(0x564162ff0c00 msgr2=0x5641630dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 shutdown_connections 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/3670036861 wait complete. 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564163442800 msgr2=0x5641634f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564163442800 0x5641634f8580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564163442000 msgr2=0x5641634f8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564163442000 0x5641634f8000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641634e3400 msgr2=0x5641634f9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641634e3400 0x5641634f9080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641634e2800 msgr2=0x5641634f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641634e2800 0x5641634f8b00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641634e2000 msgr2=0x5641632db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641634e2000 0x5641632db700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5641633e5320 tx=0x564163467320 comp rx=0 tx=0).stop 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 msgr2=0x5641632da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f771615b700 1 -- 172.21.15.191:0/742910200 reap_dead start 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x564163457260 tx=0x5641630e18c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 shutdown_connections 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/742910200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564163443400 0x5641632da680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 >> 172.21.15.191:0/742910200 conn(0x564163443000 msgr2=0x5641630de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 shutdown_connections 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/742910200 wait complete. 2024-04-27T04:34:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337c000 msgr2=0x5641632d9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337c000 0x5641632d9080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x564163e7ef30 tx=0x56416333c240 comp rx=0 tx=0).stop 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 msgr2=0x5641632d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x564163dc1e30 tx=0x5641630e1920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 shutdown_connections 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56416337c000 0x5641632d9080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 --2- 172.21.15.191:0/3249402841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564162ff0800 0x5641632d8580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.584+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 >> 172.21.15.191:0/3249402841 conn(0x564162ff0400 msgr2=0x5641630dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.585+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 shutdown_connections 2024-04-27T04:34:42.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.585+0000 7f77181b1a80 1 -- 172.21.15.191:0/3249402841 wait complete. 2024-04-27T04:34:42.592 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T04:34:42.592 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T04:34:42.592 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T04:34:42.592 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T04:34:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.638 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.636+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.636+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee4128aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee4128aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564e0be45500 con 0x564e0bb1ac00 2024-04-27T04:34:42.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee4128aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x564e0bbbca00 con 0x564e0bb1a800 2024-04-27T04:34:42.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3f234700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3ea33700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x564e0bb1ac00 0x564e0bc06800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55236/0 (socket says 172.21.15.191:55236) 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3ea33700 1 -- 172.21.15.191:0/223732804 learned_addr learned my addr 172.21.15.191:0/223732804 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3f234700 1 -- 172.21.15.191:0/223732804 >> v1:172.21.15.138:6789/0 conn(0x564e0bb1ac00 legacy=0x564e0bc06800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3f234700 1 -- 172.21.15.191:0/223732804 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb43d40 con 0x564e0bb1a800 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3e232700 1 -- 172.21.15.191:0/223732804 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432168937 0 0) 0x564e0c8e8000 con 0x564e0bb1ac00 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3f234700 1 --2- 172.21.15.191:0/223732804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x564e0bdfd9e0 tx=0x564e0bc0b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7091332657273857 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3e232700 1 -- 172.21.15.191:0/223732804 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd34a0 con 0x564e0bb1a800 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3e232700 1 -- 172.21.15.191:0/223732804 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0be45800 con 0x564e0bb1a800 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.637+0000 7fee3e232700 1 -- 172.21.15.191:0/223732804 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd3600 con 0x564e0bb1a800 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 -- 172.21.15.191:0/223732804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 msgr2=0x564e0be02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 --2- 172.21.15.191:0/223732804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x564e0bdfd9e0 tx=0x564e0bc0b920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 -- 172.21.15.191:0/223732804 shutdown_connections 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 --2- 172.21.15.191:0/223732804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 -- 172.21.15.191:0/223732804 >> 172.21.15.191:0/223732804 conn(0x564e0bb1a400 msgr2=0x564e0bc06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 -- 172.21.15.191:0/223732804 shutdown_connections 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.638+0000 7fee4128aa80 1 -- 172.21.15.191:0/223732804 wait complete. 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee4128aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee4128aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0bbbc500 con 0x564e0bb1a800 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3f234700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3f234700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38154/0 (socket says 172.21.15.191:38154) 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3f234700 1 -- 172.21.15.191:0/3498570917 learned_addr learned my addr 172.21.15.191:0/3498570917 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3f234700 1 -- 172.21.15.191:0/3498570917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb43ba0 con 0x564e0bb1a800 2024-04-27T04:34:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3f234700 1 --2- 172.21.15.191:0/3498570917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x564e0c8eb800 tx=0x564e0bc0b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9cac6a058682e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3da31700 1 -- 172.21.15.191:0/3498570917 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd3760 con 0x564e0bb1a800 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3da31700 1 -- 172.21.15.191:0/3498570917 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0be45b00 con 0x564e0bb1a800 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.639+0000 7fee3da31700 1 -- 172.21.15.191:0/3498570917 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd31e0 con 0x564e0bb1a800 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- 172.21.15.191:0/3498570917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 msgr2=0x564e0be02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3498570917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x564e0c8eb800 tx=0x564e0bc0b800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- 172.21.15.191:0/3498570917 shutdown_connections 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3498570917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- 172.21.15.191:0/3498570917 >> 172.21.15.191:0/3498570917 conn(0x564e0bb1a400 msgr2=0x564e0bc07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- 172.21.15.191:0/3498570917 shutdown_connections 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- 172.21.15.191:0/3498570917 wait complete. 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee4128aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0bbbc3c0 con 0x564e0bb1a800 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.640+0000 7fee3f234700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3f234700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38160/0 (socket says 172.21.15.191:38160) 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3f234700 1 -- 172.21.15.191:0/1873191356 learned_addr learned my addr 172.21.15.191:0/1873191356 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3f234700 1 -- 172.21.15.191:0/1873191356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb43a00 con 0x564e0bb1a800 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3f234700 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x564e0c8ebec0 tx=0x564e0bc0b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3c22e700 1 -- 172.21.15.191:0/1873191356 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd34a0 con 0x564e0bb1a800 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3c22e700 1 -- 172.21.15.191:0/1873191356 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0be45c80 con 0x564e0bb1a800 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564e0bb43860 con 0x564e0bb1a800 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee3c22e700 1 -- 172.21.15.191:0/1873191356 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd3600 con 0x564e0bb1a800 2024-04-27T04:34:42.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.641+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e0bb436c0 con 0x564e0bb1a800 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3c22e700 1 -- 172.21.15.191:0/1873191356 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564e0c8bcb00 con 0x564e0bb1a800 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee4128aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee4128aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0bb56640 con 0x564e0bb1b400 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3c22e700 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2000 0x564e0be03080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3c22e700 1 -- 172.21.15.191:0/1873191356 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564e0bb95180 con 0x564e0bb1a800 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3fa35700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3fa35700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38170/0 (socket says 172.21.15.191:38170) 2024-04-27T04:34:42.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3fa35700 1 -- 172.21.15.191:0/3852787395 learned_addr learned my addr 172.21.15.191:0/3852787395 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3f234700 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2000 0x564e0be03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3fa35700 1 -- 172.21.15.191:0/3852787395 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb42000 con 0x564e0bb1b400 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3f234700 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2000 0x564e0be03080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x564e0c995050 tx=0x564e0be6ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.642+0000 7fee3fa35700 1 --2- 172.21.15.191:0/3852787395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x564e0c994900 tx=0x564e0be6ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43925a596cbfc84 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee39a29700 1 -- 172.21.15.191:0/3852787395 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd34a0 con 0x564e0bb1b400 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee39a29700 1 -- 172.21.15.191:0/3852787395 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0bbbbc80 con 0x564e0bb1b400 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee39a29700 1 -- 172.21.15.191:0/3852787395 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd3600 con 0x564e0bb1b400 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- 172.21.15.191:0/3852787395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 msgr2=0x564e0be02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3852787395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x564e0c994900 tx=0x564e0be6ae40 comp rx=0 tx=0).stop 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- 172.21.15.191:0/3852787395 shutdown_connections 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3852787395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- 172.21.15.191:0/3852787395 >> 172.21.15.191:0/3852787395 conn(0x564e0bb1ac00 msgr2=0x564e0bc07800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- 172.21.15.191:0/3852787395 shutdown_connections 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- 172.21.15.191:0/3852787395 wait complete. 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.643+0000 7fee4128aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0bb568c0 con 0x564e0bb1b400 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee3fa35700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee3fa35700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38180/0 (socket says 172.21.15.191:38180) 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 learned_addr learned my addr 172.21.15.191:0/3820522526 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb42ea0 con 0x564e0bb1b400 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee3fa35700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x564e0c994210 tx=0x564e0be6a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee38226700 1 -- 172.21.15.191:0/3820522526 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd3760 con 0x564e0bb1b400 2024-04-27T04:34:42.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee38226700 1 -- 172.21.15.191:0/3820522526 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0bbbbb00 con 0x564e0bb1b400 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564e0bb42820 con 0x564e0bb1b400 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee38226700 1 -- 172.21.15.191:0/3820522526 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bbd34a0 con 0x564e0bb1b400 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.644+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e0bb429c0 con 0x564e0bb1b400 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee38226700 1 -- 172.21.15.191:0/3820522526 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564e0c8bcdc0 con 0x564e0bb1b400 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee38226700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2c00 0x564e0be03600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee38226700 1 -- 172.21.15.191:0/3820522526 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564e0bb95340 con 0x564e0bb1b400 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee3ea33700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2c00 0x564e0be03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bb1b800 0x564e0be03b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bb1bc00 con 0x564e0bb1b800 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee3fa35700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bb1b800 0x564e0be03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee3ea33700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2c00 0x564e0be03600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x564e0c8ea030 tx=0x564e0bf540c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.645+0000 7fee3fa35700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bb1b800 0x564e0be03b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 realm 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3000 con 0x564e0bb1b800 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3400 con 0x564e0bb1b800 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3800 con 0x564e0bb1b800 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0bea3c00 0x564e0be04100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.646+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3000 con 0x564e0bea3c00 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee3f234700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0bea3c00 0x564e0be04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee3f234700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0bea3c00 0x564e0be04100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee4128aa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee4128aa80 20 searching for the correct realm 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3400 con 0x564e0bea3c00 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee3f234700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0bea3400 0x564e0be04c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.647+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3000 con 0x564e0bea3400 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3ea33700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0bea3400 0x564e0be04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3ea33700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0bea3400 0x564e0be04c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3ea33700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bea3000 0x564e0be05180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3fa35700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bea3000 0x564e0be05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.648+0000 7fee3fa35700 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bea3000 0x564e0be05180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.649+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3400 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.650+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3400 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.651+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.652+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3400 2024-04-27T04:34:42.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.653+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.654+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.654+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.654+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.654+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.654+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.654+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3400 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3c00 2024-04-27T04:34:42.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3400 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee4128aa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee4128aa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee4128aa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee4128aa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.655+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bb1bc00 con 0x564e0bb1b800 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c400 con 0x564e0bb1b800 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c800 con 0x564e0bea3c00 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 zone default found 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 4 Realm: () 2024-04-27T04:34:42.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 zonegroup default 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6cc00 con 0x564e0bea3000 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.656+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d000 con 0x564e0bea3c00 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d400 con 0x564e0bea3c00 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 started sync module instance, tier type = 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.657+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0bea3400 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.658+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.658+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94893206641408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c800 con 0x564e0bea3400 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.658+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d000 con 0x564e0bb1b800 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.659+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.659+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94893206642560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d400 con 0x564e0bb1b800 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.659+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c400 con 0x564e0bea3c00 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.661+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94893206642560] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.661+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94893206641408] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.661+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.661+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94893206643712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6cc00 con 0x564e0bea3c00 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.662+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bb1bc00 con 0x564e0bea3400 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94893206643712] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94893206644864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bea3800 con 0x564e0bea3400 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bb1b800 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94893206646016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0bb1b800 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.663+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c800 con 0x564e0bea3000 2024-04-27T04:34:42.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.664+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94893206644864] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.664+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.664+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94893206647168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6dc00 con 0x564e0bea3000 2024-04-27T04:34:42.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.664+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d000 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.664+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94893206646016] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.665+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94893206647168] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.665+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.665+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94893206648320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d400 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.665+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c000 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.666+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94893206648320] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94893206649472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0bea3000 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=0 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=1 2024-04-27T04:34:42.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=2 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=3 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=4 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=5 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.668+0000 7fee4128aa80 20 add_watcher() i=6 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.669+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94893206649472] v31'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.669+0000 7fee4128aa80 20 add_watcher() i=7 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.669+0000 7fee4128aa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.669+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c800 con 0x564e0bb1b800 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.669+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.669+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94893208207360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6dc00 con 0x564e0bb1b800 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.670+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94893208207360] v31'47 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.670+0000 7fee26202700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee4128aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee4128aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee4128aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee4128aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0bbbc3c0 con 0x564e0bf6d400 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee3ea33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee3ea33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38188/0 (socket says 172.21.15.191:38188) 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee3ea33700 1 -- 172.21.15.191:0/4031970092 learned_addr learned my addr 172.21.15.191:0/4031970092 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee3ea33700 1 -- 172.21.15.191:0/4031970092 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb42b60 con 0x564e0bf6d400 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.671+0000 7fee3ea33700 1 --2- 172.21.15.191:0/4031970092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564e0befd800 tx=0x564e0be6bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab171df54905c481 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee25a01700 1 -- 172.21.15.191:0/4031970092 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bf8c000 con 0x564e0bf6d400 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee25a01700 1 -- 172.21.15.191:0/4031970092 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0bf8e000 con 0x564e0bf6d400 2024-04-27T04:34:42.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee25a01700 1 -- 172.21.15.191:0/4031970092 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bf8c160 con 0x564e0bf6d400 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 -- 172.21.15.191:0/4031970092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 msgr2=0x564e0be04680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 --2- 172.21.15.191:0/4031970092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564e0befd800 tx=0x564e0be6bf80 comp rx=0 tx=0).stop 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 -- 172.21.15.191:0/4031970092 shutdown_connections 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 --2- 172.21.15.191:0/4031970092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 -- 172.21.15.191:0/4031970092 >> 172.21.15.191:0/4031970092 conn(0x564e0bf6d000 msgr2=0x564e0bc08000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 -- 172.21.15.191:0/4031970092 shutdown_connections 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.672+0000 7fee4128aa80 1 -- 172.21.15.191:0/4031970092 wait complete. 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee4128aa80 1 Processor -- start 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee4128aa80 1 -- start start 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee4128aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee4128aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0bbbc500 con 0x564e0bf6d400 2024-04-27T04:34:42.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee3ea33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee3ea33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38200/0 (socket says 172.21.15.191:38200) 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 learned_addr learned my addr 172.21.15.191:0/766034495 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0bb42d00 con 0x564e0bf6d400 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.673+0000 7fee3ea33700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x564e0bf7d200 tx=0x564e0bf544e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee241fe700 1 -- 172.21.15.191:0/766034495 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bf8c2c0 con 0x564e0bf6d400 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee241fe700 1 -- 172.21.15.191:0/766034495 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0bf8e180 con 0x564e0bf6d400 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564e0bb42340 con 0x564e0bf6d400 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee241fe700 1 -- 172.21.15.191:0/766034495 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564e0bf8c000 con 0x564e0bf6d400 2024-04-27T04:34:42.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e0bb42b60 con 0x564e0bf6d400 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee241fe700 1 -- 172.21.15.191:0/766034495 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564e0c8bdb80 con 0x564e0bf6d400 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee241fe700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0c00e000 0x564e0be05700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.674+0000 7fee241fe700 1 -- 172.21.15.191:0/766034495 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564e0bb95500 con 0x564e0bf6d400 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.675+0000 7fee3fa35700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0c00e000 0x564e0be05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.675+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bf6c000 0x564e0c024000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.675+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0bf6c000 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.675+0000 7fee3f234700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bf6c000 0x564e0c024000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.675+0000 7fee3fa35700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0c00e000 0x564e0be05700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x564e0befd4a0 tx=0x564e0bf54360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.675+0000 7fee3f234700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bf6c000 0x564e0c024000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.676+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'43 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bf6c000 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.676+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bf6c800 0x564e0c024580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.676+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00e400 con 0x564e0bf6c800 2024-04-27T04:34:42.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.676+0000 7fee3ea33700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bf6c800 0x564e0c024580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.676+0000 7fee3ea33700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bf6c800 0x564e0c024580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.677+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'19 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.677+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0c00e800 0x564e0c024b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.677+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00ec00 con 0x564e0c00e800 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.677+0000 7fee3fa35700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0c00e800 0x564e0c024b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.678+0000 7fee3fa35700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0c00e800 0x564e0c024b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.678+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.678+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00f000 con 0x564e0bf6c000 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.679+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'48 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bf6c000 2024-04-27T04:34:42.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.679+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0c00f400 0x564e0c025080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.679+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00f800 con 0x564e0c00f400 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.680+0000 7fee3f234700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0c00f400 0x564e0c025080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.680+0000 7fee3f234700 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0c00f400 0x564e0c025080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.681+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'35 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.681+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00fc00 con 0x564e0c00e800 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.682+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.682+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00f000 con 0x564e0bf6c000 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.683+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bf6c000 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.683+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c400 con 0x564e0bf6c800 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.684+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'27 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.684+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6cc00 con 0x564e0c00f400 2024-04-27T04:34:42.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.685+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.685+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0c00e800 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.686+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.686+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054000 con 0x564e0c00f400 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.687+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'36 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.687+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054400 con 0x564e0bf6c800 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.688+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.688+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054800 con 0x564e0c00f400 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.689+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.689+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054c00 con 0x564e0bf6c800 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.690+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'37 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.690+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055000 con 0x564e0bf6c000 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.691+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'49 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bf6c000 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.691+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055400 con 0x564e0c00e800 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.691+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'29 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.691+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055800 con 0x564e0c00f400 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.692+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'37 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.692+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055c00 con 0x564e0c00e800 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.693+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'25 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.693+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054000 con 0x564e0bf6c000 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.694+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bf6c000 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.694+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054800 con 0x564e0bf6c000 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.695+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'50 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bf6c000 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.695+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054400 con 0x564e0c00e800 2024-04-27T04:34:42.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.696+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'30 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.696+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6cc00 con 0x564e0c00f400 2024-04-27T04:34:42.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.697+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'29 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.697+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c400 con 0x564e0c00e800 2024-04-27T04:34:42.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.698+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.698+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00f800 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.699+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'31 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.699+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00f000 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.700+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.700+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00e400 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.701+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'32 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.701+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bb1bc00 con 0x564e0bf6c800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.702+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.702+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055400 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.703+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'25 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.703+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055c00 con 0x564e0bf6c800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.704+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'28 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.704+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054400 con 0x564e0c00e800 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.705+0000 7fee3fa35700 1 -- 172.21.15.191:0/766034495 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'31 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0c00e800 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.705+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6dc00 con 0x564e0bf6c800 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.706+0000 7fee3ea33700 1 -- 172.21.15.191:0/766034495 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'38 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bf6c800 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.706+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0c00f400 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.707+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0c00f400 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.708+0000 7fee219f9700 20 reqs_thread_entry: start 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.708+0000 7fee4128aa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.708+0000 7fee4128aa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.708+0000 7fee1f9f5700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:12 2024 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee1f1d3700 20 reqs_thread_entry: start 2024-04-27T04:34:42.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee4128aa80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee4128aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c00fc00 con 0x564e0bb1b800 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee4128aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee4128aa80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.709+0000 7fee4128aa80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.710+0000 7fee26202700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.710+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94893208207360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6cc00 con 0x564e0bb1b800 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.711+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94893208207360] v31'51 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.711+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94893206641408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0bea3400 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.712+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94893206641408] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.712+0000 7fee4128aa80 20 remove_watcher() i=0 2024-04-27T04:34:42.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.712+0000 7fee4128aa80 2 removed watcher, disabling cache 2024-04-27T04:34:42.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.712+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94893206642560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055000 con 0x564e0bb1b800 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.713+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94893206642560] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.713+0000 7fee4128aa80 20 remove_watcher() i=1 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.713+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94893206643712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c055800 con 0x564e0bea3c00 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.714+0000 7fee3f234700 1 -- 172.21.15.191:0/3820522526 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94893206643712] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63200 con 0x564e0bea3c00 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.714+0000 7fee4128aa80 20 remove_watcher() i=2 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.714+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94893206644864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054000 con 0x564e0bea3400 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.715+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94893206644864] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf63440 con 0x564e0bea3400 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.715+0000 7fee4128aa80 20 remove_watcher() i=3 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.715+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94893206646016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0c054800 con 0x564e0bb1b800 2024-04-27T04:34:42.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.716+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94893206646016] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bb1b800 2024-04-27T04:34:42.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.716+0000 7fee4128aa80 20 remove_watcher() i=4 2024-04-27T04:34:42.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.716+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94893206647168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6dc00 con 0x564e0bea3000 2024-04-27T04:34:42.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.717+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94893206647168] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.717+0000 7fee4128aa80 20 remove_watcher() i=5 2024-04-27T04:34:42.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.717+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94893206648320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6d800 con 0x564e0bea3000 2024-04-27T04:34:42.724 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T04:34:42.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.718+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94893206648320] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.718+0000 7fee4128aa80 20 remove_watcher() i=6 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.718+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94893206649472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564e0bf6c400 con 0x564e0bea3000 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee3fa35700 1 -- 172.21.15.191:0/3820522526 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94893206649472] v31'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564e0bf62b40 con 0x564e0bea3000 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 20 remove_watcher() i=7 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bea3000 msgr2=0x564e0be05180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bea3000 0x564e0be05180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bb1b800 msgr2=0x564e0be03b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bb1b800 0x564e0be03b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0bea3400 msgr2=0x564e0be04c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0bea3400 0x564e0be04c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0bea3c00 msgr2=0x564e0be04100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0bea3c00 0x564e0be04100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2c00 msgr2=0x564e0be03600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2c00 0x564e0be03600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x564e0c8ea030 tx=0x564e0bf540c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 msgr2=0x564e0be02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x564e0c994210 tx=0x564e0be6a6c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee3ea33700 1 -- 172.21.15.191:0/3820522526 reap_dead start 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 shutdown_connections 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 --2- 172.21.15.191:0/3820522526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1b400 0x564e0be02b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 >> 172.21.15.191:0/3820522526 conn(0x564e0bb1ac00 msgr2=0x564e0bc07800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 shutdown_connections 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.719+0000 7fee4128aa80 1 -- 172.21.15.191:0/3820522526 wait complete. 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bf6c800 msgr2=0x564e0c024580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564e0bf6c800 0x564e0c024580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bf6c000 msgr2=0x564e0c024000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564e0bf6c000 0x564e0c024000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0c00f400 msgr2=0x564e0c025080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564e0c00f400 0x564e0c025080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0c00e800 msgr2=0x564e0c024b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564e0c00e800 0x564e0c024b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0c00e000 msgr2=0x564e0be05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0c00e000 0x564e0be05700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x564e0befd4a0 tx=0x564e0bf54360 comp rx=0 tx=0).stop 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 msgr2=0x564e0be04680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee3f234700 1 -- 172.21.15.191:0/766034495 reap_dead start 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x564e0bf7d200 tx=0x564e0bf544e0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 shutdown_connections 2024-04-27T04:34:42.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/766034495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bf6d400 0x564e0be04680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 >> 172.21.15.191:0/766034495 conn(0x564e0bf6d000 msgr2=0x564e0bc08000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 shutdown_connections 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/766034495 wait complete. 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2000 msgr2=0x564e0be03080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2000 0x564e0be03080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x564e0c995050 tx=0x564e0be6ab40 comp rx=0 tx=0).stop 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 msgr2=0x564e0be02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x564e0c8ebec0 tx=0x564e0bc0b920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 shutdown_connections 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564e0bea2000 0x564e0be03080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 --2- 172.21.15.191:0/1873191356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e0bb1a800 0x564e0be02580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 >> 172.21.15.191:0/1873191356 conn(0x564e0bb1a400 msgr2=0x564e0bc07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 shutdown_connections 2024-04-27T04:34:42.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.720+0000 7fee4128aa80 1 -- 172.21.15.191:0/1873191356 wait complete. 2024-04-27T04:34:42.729 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T04:34:42.729 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T04:34:42.729 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-27T04:34:42.777 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.777 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6f5b6a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56380a024800 0x56380a30c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6f5b6a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56380a34f500 con 0x56380a024c00 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6f5b6a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x56380a0c6a00 con 0x56380a024800 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6d560700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56380a024800 0x56380a30c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6cd5f700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x56380a024c00 0x56380a110800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55238/0 (socket says 172.21.15.191:55238) 2024-04-27T04:34:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.776+0000 7f3e6cd5f700 1 -- 172.21.15.191:0/681664198 learned_addr learned my addr 172.21.15.191:0/681664198 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6d560700 1 -- 172.21.15.191:0/681664198 >> v1:172.21.15.138:6789/0 conn(0x56380a024c00 legacy=0x56380a110800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6d560700 1 -- 172.21.15.191:0/681664198 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04dd40 con 0x56380a024800 2024-04-27T04:34:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6d560700 1 --2- 172.21.15.191:0/681664198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x56380a3079e0 tx=0x56380a115920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be5a437d38f4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6c55e700 1 -- 172.21.15.191:0/681664198 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd4a0 con 0x56380a024800 2024-04-27T04:34:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6c55e700 1 -- 172.21.15.191:0/681664198 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a34f800 con 0x56380a024800 2024-04-27T04:34:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6c55e700 1 -- 172.21.15.191:0/681664198 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd600 con 0x56380a024800 2024-04-27T04:34:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/681664198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 msgr2=0x56380a30c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/681664198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x56380a3079e0 tx=0x56380a115920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/681664198 shutdown_connections 2024-04-27T04:34:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/681664198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/681664198 >> 172.21.15.191:0/681664198 conn(0x56380a024400 msgr2=0x56380a110000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.777+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/681664198 shutdown_connections 2024-04-27T04:34:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/681664198 wait complete. 2024-04-27T04:34:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6f5b6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6f5b6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380a0c6500 con 0x56380a024800 2024-04-27T04:34:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6d560700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6d560700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38208/0 (socket says 172.21.15.191:38208) 2024-04-27T04:34:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.778+0000 7f3e6d560700 1 -- 172.21.15.191:0/1341166017 learned_addr learned my addr 172.21.15.191:0/1341166017 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6d560700 1 -- 172.21.15.191:0/1341166017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04dba0 con 0x56380a024800 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6d560700 1 --2- 172.21.15.191:0/1341166017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x56380adfd740 tx=0x56380a115800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8419af4dd423a7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6bd5d700 1 -- 172.21.15.191:0/1341166017 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd760 con 0x56380a024800 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6bd5d700 1 -- 172.21.15.191:0/1341166017 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a34fb00 con 0x56380a024800 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6bd5d700 1 -- 172.21.15.191:0/1341166017 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd1e0 con 0x56380a024800 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1341166017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 msgr2=0x56380a30c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1341166017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x56380adfd740 tx=0x56380a115800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1341166017 shutdown_connections 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1341166017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1341166017 >> 172.21.15.191:0/1341166017 conn(0x56380a024400 msgr2=0x56380a111000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1341166017 shutdown_connections 2024-04-27T04:34:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.779+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1341166017 wait complete. 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6f5b6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6f5b6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380a0c63c0 con 0x56380a024800 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6d560700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6d560700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38210/0 (socket says 172.21.15.191:38210) 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6d560700 1 -- 172.21.15.191:0/1844006581 learned_addr learned my addr 172.21.15.191:0/1844006581 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6d560700 1 -- 172.21.15.191:0/1844006581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04da00 con 0x56380a024800 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6d560700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56380adfd5f0 tx=0x56380a115920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6a55a700 1 -- 172.21.15.191:0/1844006581 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd4a0 con 0x56380a024800 2024-04-27T04:34:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.780+0000 7f3e6a55a700 1 -- 172.21.15.191:0/1844006581 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a34fc80 con 0x56380a024800 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56380a04d860 con 0x56380a024800 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6a55a700 1 -- 172.21.15.191:0/1844006581 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd600 con 0x56380a024800 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56380a04d6c0 con 0x56380a024800 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6a55a700 1 -- 172.21.15.191:0/1844006581 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56380adc6b00 con 0x56380a024800 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6f5b6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6f5b6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380a0c6000 con 0x56380a025400 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6a55a700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4000 0x56380a30d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.781+0000 7f3e6a55a700 1 -- 172.21.15.191:0/1844006581 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56380a09f180 con 0x56380a024800 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38212/0 (socket says 172.21.15.191:38212) 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6dd61700 1 -- 172.21.15.191:0/2858145565 learned_addr learned my addr 172.21.15.191:0/2858145565 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6d560700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4000 0x56380a30d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6dd61700 1 -- 172.21.15.191:0/2858145565 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04c000 con 0x56380a025400 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6d560700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4000 0x56380a30d080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x56380aeb6ed0 tx=0x56380a378c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e6dd61700 1 --2- 172.21.15.191:0/2858145565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56380aeb6210 tx=0x56380a378b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133d20c4d7709d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e67d55700 1 -- 172.21.15.191:0/2858145565 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd4a0 con 0x56380a025400 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e67d55700 1 -- 172.21.15.191:0/2858145565 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a0c5c80 con 0x56380a025400 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.782+0000 7f3e67d55700 1 -- 172.21.15.191:0/2858145565 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd600 con 0x56380a025400 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2858145565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 msgr2=0x56380a30cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/2858145565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56380aeb6210 tx=0x56380a378b40 comp rx=0 tx=0).stop 2024-04-27T04:34:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2858145565 shutdown_connections 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/2858145565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2858145565 >> 172.21.15.191:0/2858145565 conn(0x56380a024c00 msgr2=0x56380a111800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2858145565 shutdown_connections 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2858145565 wait complete. 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.783+0000 7f3e6f5b6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380a060640 con 0x56380a025400 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38214/0 (socket says 172.21.15.191:38214) 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e6dd61700 1 -- 172.21.15.191:0/3274387807 learned_addr learned my addr 172.21.15.191:0/3274387807 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e6dd61700 1 -- 172.21.15.191:0/3274387807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04cea0 con 0x56380a025400 2024-04-27T04:34:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e6dd61700 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56380aeb65d0 tx=0x56380a378a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e66552700 1 -- 172.21.15.191:0/3274387807 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd760 con 0x56380a025400 2024-04-27T04:34:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e66552700 1 -- 172.21.15.191:0/3274387807 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a0c5b00 con 0x56380a025400 2024-04-27T04:34:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e66552700 1 -- 172.21.15.191:0/3274387807 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd4a0 con 0x56380a025400 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.784+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56380a04c820 con 0x56380a025400 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.785+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56380a04c9c0 con 0x56380a025400 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6f5b6a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6f5b6a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6f5b6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6f5b6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380a0608c0 con 0x56380a025c00 2024-04-27T04:34:42.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e66552700 1 -- 172.21.15.191:0/3274387807 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56380adc6dc0 con 0x56380a025400 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38222/0 (socket says 172.21.15.191:38222) 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e6dd61700 1 -- 172.21.15.191:0/2837360129 learned_addr learned my addr 172.21.15.191:0/2837360129 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e66552700 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4c00 0x56380a30e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.786+0000 7f3e66552700 1 -- 172.21.15.191:0/3274387807 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56380a09f340 con 0x56380a025400 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6dd61700 1 -- 172.21.15.191:0/2837360129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04cb60 con 0x56380a025c00 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6d560700 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4c00 0x56380a30e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6dd61700 1 --2- 172.21.15.191:0/2837360129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x56380a3f48a0 tx=0x56380a379500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2f6a5a7524d003 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e5452e700 1 -- 172.21.15.191:0/2837360129 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd600 con 0x56380a025c00 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e5452e700 1 -- 172.21.15.191:0/2837360129 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a46c000 con 0x56380a025c00 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e5452e700 1 -- 172.21.15.191:0/2837360129 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd1e0 con 0x56380a025c00 2024-04-27T04:34:42.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6d560700 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4c00 0x56380a30e680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x56380aeb6f90 tx=0x56380a378de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2837360129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 msgr2=0x56380a30d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/2837360129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x56380a3f48a0 tx=0x56380a379500 comp rx=0 tx=0).stop 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2837360129 shutdown_connections 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/2837360129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.787+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2837360129 >> 172.21.15.191:0/2837360129 conn(0x56380a025800 msgr2=0x56380a112000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2837360129 shutdown_connections 2024-04-27T04:34:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/2837360129 wait complete. 2024-04-27T04:34:42.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6f5b6a80 1 Processor -- start 2024-04-27T04:34:42.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6f5b6a80 1 -- start start 2024-04-27T04:34:42.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6f5b6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6f5b6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380a060f00 con 0x56380a025c00 2024-04-27T04:34:42.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6dd61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38230/0 (socket says 172.21.15.191:38230) 2024-04-27T04:34:42.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.788+0000 7f3e6dd61700 1 -- 172.21.15.191:0/1885300665 learned_addr learned my addr 172.21.15.191:0/1885300665 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e6dd61700 1 -- 172.21.15.191:0/1885300665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380a04cd00 con 0x56380a025c00 2024-04-27T04:34:42.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e6dd61700 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x56380a3f5cb0 tx=0x56380a46a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e52d2b700 1 -- 172.21.15.191:0/1885300665 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd600 con 0x56380a025c00 2024-04-27T04:34:42.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e52d2b700 1 -- 172.21.15.191:0/1885300665 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56380a46c180 con 0x56380a025c00 2024-04-27T04:34:42.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e52d2b700 1 -- 172.21.15.191:0/1885300665 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56380a0dd1e0 con 0x56380a025c00 2024-04-27T04:34:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56380a04c340 con 0x56380a025c00 2024-04-27T04:34:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.789+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56380a478000 con 0x56380a025c00 2024-04-27T04:34:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e52d2b700 1 -- 172.21.15.191:0/1885300665 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56380adc7080 con 0x56380a025c00 2024-04-27T04:34:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e52d2b700 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a4de000 0x56380a30ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e52d2b700 1 -- 172.21.15.191:0/1885300665 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56380a09f500 con 0x56380a025c00 2024-04-27T04:34:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e50d27700 20 reqs_thread_entry: start 2024-04-27T04:34:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56380a3b5c00 0x56380a30db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56380a4de400 con 0x56380a3b5c00 2024-04-27T04:34:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e6cd5f700 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a4de000 0x56380a30ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.790+0000 7f3e6dd61700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56380a3b5c00 0x56380a30db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6dd61700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56380a3b5c00 0x56380a30db80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6cd5f700 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a4de000 0x56380a30ec00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x56380a3f5fb0 tx=0x56380a46b740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6dd61700 1 -- 172.21.15.191:0/1844006581 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56380a504000 con 0x56380a3b5c00 2024-04-27T04:34:42.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56380a4de800 0x56380a30e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:2 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56380a4dec00 con 0x56380a4de800 2024-04-27T04:34:42.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6d560700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56380a4de800 0x56380a30e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6d560700 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56380a4de800 0x56380a30e100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.791+0000 7f3e6d560700 1 -- 172.21.15.191:0/1844006581 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(2 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [call out=24b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+898 (crc 0 0 0) 0x56380a504240 con 0x56380a4de800 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.792+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [stat,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56380a4df000 con 0x56380a4de800 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6d560700 1 -- 172.21.15.191:0/1844006581 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(3 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [stat,call,writefull 0~874] v31'2 uv2 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x56380a504240 con 0x56380a4de800 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4c00 msgr2=0x56380a30e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4c00 0x56380a30e680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x56380aeb6f90 tx=0x56380a378de0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 msgr2=0x56380a30cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56380aeb65d0 tx=0x56380a378a20 comp rx=0 tx=0).stop 2024-04-27T04:34:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 shutdown_connections 2024-04-27T04:34:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4c00 0x56380a30e680 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/3274387807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025400 0x56380a30cb00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 >> 172.21.15.191:0/3274387807 conn(0x56380a024c00 msgr2=0x56380a111800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 shutdown_connections 2024-04-27T04:34:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.793+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/3274387807 wait complete. 2024-04-27T04:34:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a4de000 msgr2=0x56380a30ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a4de000 0x56380a30ec00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x56380a3f5fb0 tx=0x56380a46b740 comp rx=0 tx=0).stop 2024-04-27T04:34:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 msgr2=0x56380a30d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x56380a3f5cb0 tx=0x56380a46a420 comp rx=0 tx=0).stop 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 shutdown_connections 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a4de000 0x56380a30ec00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1885300665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a025c00 0x56380a30d600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 >> 172.21.15.191:0/1885300665 conn(0x56380a025800 msgr2=0x56380a112000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 shutdown_connections 2024-04-27T04:34:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1885300665 wait complete. 2024-04-27T04:34:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56380a3b5c00 msgr2=0x56380a30db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56380a3b5c00 0x56380a30db80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56380a4de800 msgr2=0x56380a30e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56380a4de800 0x56380a30e100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4000 msgr2=0x56380a30d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4000 0x56380a30d080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x56380aeb6ed0 tx=0x56380a378c60 comp rx=0 tx=0).stop 2024-04-27T04:34:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 msgr2=0x56380a30c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56380adfd5f0 tx=0x56380a115920 comp rx=0 tx=0).stop 2024-04-27T04:34:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 shutdown_connections 2024-04-27T04:34:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56380a4de800 0x56380a30e100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56380a3b5c00 0x56380a30db80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380a024800 0x56380a30c580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 --2- 172.21.15.191:0/1844006581 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56380a3b4000 0x56380a30d080 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 >> 172.21.15.191:0/1844006581 conn(0x56380a024400 msgr2=0x56380a111000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 shutdown_connections 2024-04-27T04:34:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.794+0000 7f3e6f5b6a80 1 -- 172.21.15.191:0/1844006581 wait complete. 2024-04-27T04:34:42.816 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:34:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:34:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T04:34:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "control_pool": "default.rgw.control", 2024-04-27T04:34:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T04:34:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T04:34:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "log_pool": "default.rgw.log", 2024-04-27T04:34:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T04:34:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T04:34:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T04:34:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T04:34:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T04:34:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T04:34:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T04:34:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T04:34:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T04:34:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "system_key": { 2024-04-27T04:34:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "", 2024-04-27T04:34:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "" 2024-04-27T04:34:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_pools": [ 2024-04-27T04:34:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "default-placement", 2024-04-27T04:34:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:34:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T04:34:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:34:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:34:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T04:34:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:34:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:34:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": true 2024-04-27T04:34:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "realm_id": "", 2024-04-27T04:34:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T04:34:42.824 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:42.824 DEBUG:tasks.util.rgw: json result: {'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T04:34:42.827 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-27T04:34:42.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-27T04:34:42.827 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-27T04:34:42.894 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:42.894 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:42.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.870+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.870+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.870+0000 7ff2f356ba80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.870+0000 7ff2f356ba80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d2e5c33500 con 0x55d2e5908c00 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.870+0000 7ff2f356ba80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d2e59aaa00 con 0x55d2e5908800 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0d14700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55d2e5908c00 0x55d2e59f4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55250/0 (socket says 172.21.15.191:55250) 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0d14700 1 -- 172.21.15.191:0/1929405619 learned_addr learned my addr 172.21.15.191:0/1929405619 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f1515700 1 --2- 172.21.15.191:0/1929405619 >> v2:172.21.15.138:3300/0 conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0513700 1 -- 172.21.15.191:0/1929405619 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747629160 0 0) 0x55d2e66d6000 con 0x55d2e5908c00 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0513700 1 -- 172.21.15.191:0/1929405619 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55d2e5c33680 con 0x55d2e5908c00 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f1515700 1 -- 172.21.15.191:0/1929405619 >> v1:172.21.15.138:6789/0 conn(0x55d2e5908c00 legacy=0x55d2e59f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f1515700 1 -- 172.21.15.191:0/1929405619 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5931d40 con 0x55d2e5908800 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f1515700 1 --2- 172.21.15.191:0/1929405619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55d2e5beb9e0 tx=0x55d2e59f9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1655a4804b52ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0513700 1 -- 172.21.15.191:0/1929405619 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c14a0 con 0x55d2e5908800 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0513700 1 -- 172.21.15.191:0/1929405619 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e5c33680 con 0x55d2e5908800 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.871+0000 7ff2f0513700 1 -- 172.21.15.191:0/1929405619 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c1600 con 0x55d2e5908800 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1929405619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 msgr2=0x55d2e5bf0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1929405619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55d2e5beb9e0 tx=0x55d2e59f9980 comp rx=0 tx=0).stop 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1929405619 shutdown_connections 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1929405619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1929405619 >> 172.21.15.191:0/1929405619 conn(0x55d2e5908400 msgr2=0x55d2e59f4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1929405619 shutdown_connections 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1929405619 wait complete. 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.872+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f356ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f356ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e59aa500 con 0x55d2e5908800 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f1515700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f1515700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38250/0 (socket says 172.21.15.191:38250) 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f1515700 1 -- 172.21.15.191:0/263252638 learned_addr learned my addr 172.21.15.191:0/263252638 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f1515700 1 -- 172.21.15.191:0/263252638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5931ba0 con 0x55d2e5908800 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2f1515700 1 --2- 172.21.15.191:0/263252638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55d2e66d9800 tx=0x55d2e59f9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c54128f35dd944 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2efd12700 1 -- 172.21.15.191:0/263252638 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c1760 con 0x55d2e5908800 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2efd12700 1 -- 172.21.15.191:0/263252638 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e5c33980 con 0x55d2e5908800 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.873+0000 7ff2efd12700 1 -- 172.21.15.191:0/263252638 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c14a0 con 0x55d2e5908800 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- 172.21.15.191:0/263252638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 msgr2=0x55d2e5bf0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/263252638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55d2e66d9800 tx=0x55d2e59f9800 comp rx=0 tx=0).stop 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- 172.21.15.191:0/263252638 shutdown_connections 2024-04-27T04:34:42.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/263252638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- 172.21.15.191:0/263252638 >> 172.21.15.191:0/263252638 conn(0x55d2e5908400 msgr2=0x55d2e59f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- 172.21.15.191:0/263252638 shutdown_connections 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- 172.21.15.191:0/263252638 wait complete. 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f356ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e59aa3c0 con 0x55d2e5908800 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f1515700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f1515700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38262/0 (socket says 172.21.15.191:38262) 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.874+0000 7ff2f1515700 1 -- 172.21.15.191:0/1666893853 learned_addr learned my addr 172.21.15.191:0/1666893853 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2f1515700 1 -- 172.21.15.191:0/1666893853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5931a00 con 0x55d2e5908800 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2f1515700 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55d2e66d9ec0 tx=0x55d2e59f9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2ee50f700 1 -- 172.21.15.191:0/1666893853 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c1600 con 0x55d2e5908800 2024-04-27T04:34:42.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2ee50f700 1 -- 172.21.15.191:0/1666893853 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e5c33b00 con 0x55d2e5908800 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d2e5931860 con 0x55d2e5908800 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2ee50f700 1 -- 172.21.15.191:0/1666893853 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c18c0 con 0x55d2e5908800 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2e59316c0 con 0x55d2e5908800 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.875+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f356ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f356ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e59aa000 con 0x55d2e5909400 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2ee50f700 1 -- 172.21.15.191:0/1666893853 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d2e66aab00 con 0x55d2e5908800 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1d16700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1d16700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38272/0 (socket says 172.21.15.191:38272) 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1d16700 1 -- 172.21.15.191:0/2898619122 learned_addr learned my addr 172.21.15.191:0/2898619122 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2ee50f700 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96000 0x55d2e5bf1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2ee50f700 1 -- 172.21.15.191:0/1666893853 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d2e5983180 con 0x55d2e5908800 2024-04-27T04:34:42.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1515700 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96000 0x55d2e5bf1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1d16700 1 -- 172.21.15.191:0/2898619122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5930000 con 0x55d2e5909400 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/2898619122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55d2e5c94060 tx=0x55d2e5c58b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339bca84a3ab1009 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2f1515700 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96000 0x55d2e5bf1080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55d2e6794fc0 tx=0x55d2e5c58c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2ebd0a700 1 -- 172.21.15.191:0/2898619122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c1600 con 0x55d2e5909400 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2ebd0a700 1 -- 172.21.15.191:0/2898619122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e59a9c80 con 0x55d2e5909400 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.876+0000 7ff2ebd0a700 1 -- 172.21.15.191:0/2898619122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c18c0 con 0x55d2e5909400 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- 172.21.15.191:0/2898619122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 msgr2=0x55d2e5bf0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/2898619122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55d2e5c94060 tx=0x55d2e5c58b40 comp rx=0 tx=0).stop 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- 172.21.15.191:0/2898619122 shutdown_connections 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/2898619122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- 172.21.15.191:0/2898619122 >> 172.21.15.191:0/2898619122 conn(0x55d2e5908c00 msgr2=0x55d2e59f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- 172.21.15.191:0/2898619122 shutdown_connections 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- 172.21.15.191:0/2898619122 wait complete. 2024-04-27T04:34:42.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f356ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e5944640 con 0x55d2e5909400 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f1d16700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f1d16700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38274/0 (socket says 172.21.15.191:38274) 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.877+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 learned_addr learned my addr 172.21.15.191:0/666271994 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5930ea0 con 0x55d2e5909400 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55d2e6794240 tx=0x55d2e5c585a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2ea507700 1 -- 172.21.15.191:0/666271994 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c1760 con 0x55d2e5909400 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2ea507700 1 -- 172.21.15.191:0/666271994 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e59a9b00 con 0x55d2e5909400 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d2e5930820 con 0x55d2e5909400 2024-04-27T04:34:42.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2ea507700 1 -- 172.21.15.191:0/666271994 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e59c1600 con 0x55d2e5909400 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.878+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2e59309c0 con 0x55d2e5909400 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2ea507700 1 -- 172.21.15.191:0/666271994 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d2e66aadc0 con 0x55d2e5909400 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2ea507700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96c00 0x55d2e5bf1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2ea507700 1 -- 172.21.15.191:0/666271994 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d2e5983340 con 0x55d2e5909400 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96c00 0x55d2e5bf1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d2e5909800 0x55d2e5bf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5909c00 con 0x55d2e5909800 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d2e5909800 0x55d2e5bf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96c00 0x55d2e5bf1600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55d2e66d8030 tx=0x55d2e5d460c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.879+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d2e5909800 0x55d2e5bf1b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 realm 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97000 con 0x55d2e5909800 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97400 con 0x55d2e5909800 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97800 con 0x55d2e5909800 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d2e5c97c00 0x55d2e5bf2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.880+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97000 con 0x55d2e5c97c00 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f1515700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d2e5c97c00 0x55d2e5bf2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f1515700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d2e5c97c00 0x55d2e5bf2100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f356ba80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f356ba80 20 searching for the correct realm 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97400 con 0x55d2e5c97c00 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f1515700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d2e5c97400 0x55d2e5bf2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.881+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97000 con 0x55d2e5c97400 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.882+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d2e5c97400 0x55d2e5bf2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.882+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d2e5c97400 0x55d2e5bf2c00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.882+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.882+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d2e5c97000 0x55d2e5bf3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.882+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.882+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d2e5c97000 0x55d2e5bf3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d2e5c97000 0x55d2e5bf3180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.883+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97400 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.884+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97400 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.885+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.886+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97400 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.887+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.888+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.888+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.888+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.888+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.888+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.888+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97400 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97c00 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.889+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97400 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5909c00 con 0x55d2e5909800 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e400 con 0x55d2e5909800 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e800 con 0x55d2e5c97c00 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 zone default found 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 4 Realm: () 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 zonegroup default 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.890+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5ec00 con 0x55d2e5c97000 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f000 con 0x55d2e5c97c00 2024-04-27T04:34:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f400 con 0x55d2e5c97c00 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 started sync module instance, tier type = 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.891+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f800 con 0x55d2e5c97400 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.892+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.892+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94364285958912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e800 con 0x55d2e5c97400 2024-04-27T04:34:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.892+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f000 con 0x55d2e5909800 2024-04-27T04:34:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.893+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94364285958912] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.893+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.893+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94364285960064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f400 con 0x55d2e5909800 2024-04-27T04:34:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.893+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e400 con 0x55d2e5c97c00 2024-04-27T04:34:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.894+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.894+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94364285961216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5ec00 con 0x55d2e5c97c00 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.894+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94364285960064] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.894+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5909c00 con 0x55d2e5c97400 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.895+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94364285961216] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.895+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.895+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94364285962368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5c97800 con 0x55d2e5c97400 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.895+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5909800 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.898+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.898+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94364285963520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f800 con 0x55d2e5909800 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.898+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e800 con 0x55d2e5c97000 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.899+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94364285962368] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.899+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.899+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94364285964672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5fc00 con 0x55d2e5c97000 2024-04-27T04:34:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.899+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f000 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.899+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94364285963520] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.900+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94364285964672] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.900+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.900+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94364285965824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f400 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.900+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e000 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94364285965824] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94364285966976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f800 con 0x55d2e5c97000 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=0 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=1 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=2 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=3 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=4 2024-04-27T04:34:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=5 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.901+0000 7ff2f356ba80 20 add_watcher() i=6 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.902+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94364285966976] v31'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.902+0000 7ff2f356ba80 20 add_watcher() i=7 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.902+0000 7ff2f356ba80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.902+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e800 con 0x55d2e5909800 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.903+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.903+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94364287541248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5fc00 con 0x55d2e5909800 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94364287541248] v31'52 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2d84e3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f356ba80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f356ba80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f356ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f356ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e5944780 con 0x55d2e5d5f400 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.904+0000 7ff2f0d14700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2f0d14700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38278/0 (socket says 172.21.15.191:38278) 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2f0d14700 1 -- 172.21.15.191:0/1922962828 learned_addr learned my addr 172.21.15.191:0/1922962828 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2f0d14700 1 -- 172.21.15.191:0/1922962828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5930b60 con 0x55d2e5d5f400 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/1922962828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55d2e5cf44e0 tx=0x55d2e5c59f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9b4a6c45fbd88b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2d7ce2700 1 -- 172.21.15.191:0/1922962828 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e5d7e000 con 0x55d2e5d5f400 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2d7ce2700 1 -- 172.21.15.191:0/1922962828 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e5d76180 con 0x55d2e5d5f400 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.905+0000 7ff2d7ce2700 1 -- 172.21.15.191:0/1922962828 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e5d7e160 con 0x55d2e5d5f400 2024-04-27T04:34:42.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1922962828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 msgr2=0x55d2e5bf2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1922962828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55d2e5cf44e0 tx=0x55d2e5c59f80 comp rx=0 tx=0).stop 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1922962828 shutdown_connections 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1922962828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1922962828 >> 172.21.15.191:0/1922962828 conn(0x55d2e5d5f000 msgr2=0x55d2e59f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1922962828 shutdown_connections 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1922962828 wait complete. 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 Processor -- start 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- start start 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f356ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e59aa000 con 0x55d2e5d5f400 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f0d14700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f0d14700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38288/0 (socket says 172.21.15.191:38288) 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.906+0000 7ff2f0d14700 1 -- 172.21.15.191:0/3669623537 learned_addr learned my addr 172.21.15.191:0/3669623537 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2f0d14700 1 -- 172.21.15.191:0/3669623537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e5930d00 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2f0d14700 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d2e5d81200 tx=0x55d2e5d464e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2d64df700 1 -- 172.21.15.191:0/3669623537 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e5d7e2c0 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2d64df700 1 -- 172.21.15.191:0/3669623537 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2e5d76300 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d2e5930340 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2d64df700 1 -- 172.21.15.191:0/3669623537 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d2e5d7e000 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.907+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2e5930b60 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.908+0000 7ff2d64df700 1 -- 172.21.15.191:0/3669623537 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d2e66abb80 con 0x55d2e5d5f400 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.908+0000 7ff2d64df700 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5e00000 0x55d2e5bf3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:42.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.908+0000 7ff2d64df700 1 -- 172.21.15.191:0/3669623537 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d2e5983500 con 0x55d2e5d5f400 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.908+0000 7ff2f356ba80 5 note: GC not initialized 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.908+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5e00000 0x55d2e5bf3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.908+0000 7ff2f1d16700 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5e00000 0x55d2e5bf3700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55d2e5cf5800 tx=0x55d2e5d46360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2d3cda700 20 reqs_thread_entry: start 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2f356ba80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2f356ba80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:42.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2d1cd6700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:13 2024 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2d14b4700 20 reqs_thread_entry: start 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f800 con 0x55d2e5909800 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.909+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e800 con 0x55d2e5c97400 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00400 con 0x55d2e5909800 2024-04-27T04:34:42.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00800 con 0x55d2e5c97c00 2024-04-27T04:34:42.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00c00 con 0x55d2e5c97c00 2024-04-27T04:34:42.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.910+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01000 con 0x55d2e5c97c00 2024-04-27T04:34:42.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01400 con 0x55d2e5909800 2024-04-27T04:34:42.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01800 con 0x55d2e5909800 2024-04-27T04:34:42.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01c00 con 0x55d2e5c97c00 2024-04-27T04:34:42.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00800 con 0x55d2e5c97c00 2024-04-27T04:34:42.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.911+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00c00 con 0x55d2e5c97000 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e400 con 0x55d2e5909800 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5ec00 con 0x55d2e5c97400 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5fc00 con 0x55d2e5c97c00 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.912+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f800 con 0x55d2e5c97000 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e400 con 0x55d2e5c97000 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00400 con 0x55d2e5c97c00 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01400 con 0x55d2e5c97000 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01800 con 0x55d2e5909800 2024-04-27T04:34:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.913+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00c00 con 0x55d2e5c97c00 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01000 con 0x55d2e5c97000 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01c00 con 0x55d2e5c97400 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00800 con 0x55d2e5c97000 2024-04-27T04:34:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00400 con 0x55d2e5909800 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.914+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5fc00 con 0x55d2e5909800 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01400 con 0x55d2e5c97c00 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01800 con 0x55d2e5909800 2024-04-27T04:34:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01000 con 0x55d2e5909800 2024-04-27T04:34:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.915+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00800 con 0x55d2e5c97c00 2024-04-27T04:34:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00400 con 0x55d2e5c97c00 2024-04-27T04:34:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01c00 con 0x55d2e5c97c00 2024-04-27T04:34:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5f800 con 0x55d2e5c97000 2024-04-27T04:34:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.916+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.917+0000 7ff2d84e3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.917+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94364287541248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e400 con 0x55d2e5909800 2024-04-27T04:34:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.918+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94364287541248] v31'53 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.918+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94364285958912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5fc00 con 0x55d2e5c97400 2024-04-27T04:34:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.919+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94364285958912] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.919+0000 7ff2f356ba80 20 remove_watcher() i=0 2024-04-27T04:34:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.919+0000 7ff2f356ba80 2 removed watcher, disabling cache 2024-04-27T04:34:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.919+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94364285960064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5e800 con 0x55d2e5909800 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.921+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94364285960064] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.921+0000 7ff2f356ba80 20 remove_watcher() i=1 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.921+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94364285961216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5ec00 con 0x55d2e5c97c00 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.921+0000 7ff2f1515700 1 -- 172.21.15.191:0/666271994 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94364285961216] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b200 con 0x55d2e5c97c00 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.922+0000 7ff2f356ba80 20 remove_watcher() i=2 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.922+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94364285962368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5909c00 con 0x55d2e5c97400 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.923+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94364285962368] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5b440 con 0x55d2e5c97400 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.923+0000 7ff2f356ba80 20 remove_watcher() i=3 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.923+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94364285963520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5d5ec00 con 0x55d2e5909800 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.924+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94364285963520] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5909800 2024-04-27T04:34:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.924+0000 7ff2f356ba80 20 remove_watcher() i=4 2024-04-27T04:34:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.924+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94364285964672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00c00 con 0x55d2e5c97000 2024-04-27T04:34:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.925+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94364285964672] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.925+0000 7ff2f356ba80 20 remove_watcher() i=5 2024-04-27T04:34:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.925+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94364285965824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e01400 con 0x55d2e5c97000 2024-04-27T04:34:42.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.930+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94364285965824] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.930+0000 7ff2f356ba80 20 remove_watcher() i=6 2024-04-27T04:34:42.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.930+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94364285966976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d2e5e00800 con 0x55d2e5c97000 2024-04-27T04:34:42.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f1d16700 1 -- 172.21.15.191:0/666271994 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94364285966976] v31'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d2e5d5ab40 con 0x55d2e5c97000 2024-04-27T04:34:42.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 20 remove_watcher() i=7 2024-04-27T04:34:42.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d2e5c97000 msgr2=0x55d2e5bf3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d2e5c97000 0x55d2e5bf3180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d2e5909800 msgr2=0x55d2e5bf1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d2e5909800 0x55d2e5bf1b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d2e5c97400 msgr2=0x55d2e5bf2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d2e5c97400 0x55d2e5bf2c00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d2e5c97c00 msgr2=0x55d2e5bf2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d2e5c97c00 0x55d2e5bf2100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96c00 msgr2=0x55d2e5bf1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96c00 0x55d2e5bf1600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55d2e66d8030 tx=0x55d2e5d460c0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 msgr2=0x55d2e5bf0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/666271994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5909400 0x55d2e5bf0b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55d2e6794240 tx=0x55d2e5c585a0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 reap_dead start 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 shutdown_connections 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f0d14700 1 -- 172.21.15.191:0/666271994 reap_dead start 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 >> 172.21.15.191:0/666271994 conn(0x55d2e5908c00 msgr2=0x55d2e59f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 shutdown_connections 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.931+0000 7ff2f356ba80 1 -- 172.21.15.191:0/666271994 wait complete. 2024-04-27T04:34:42.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5e00000 msgr2=0x55d2e5bf3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5e00000 0x55d2e5bf3700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55d2e5cf5800 tx=0x55d2e5d46360 comp rx=0 tx=0).stop 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 msgr2=0x55d2e5bf2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d2e5d81200 tx=0x55d2e5d464e0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 shutdown_connections 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5e00000 0x55d2e5bf3700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/3669623537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5d5f400 0x55d2e5bf2680 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 >> 172.21.15.191:0/3669623537 conn(0x55d2e5d5f000 msgr2=0x55d2e59f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 shutdown_connections 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/3669623537 wait complete. 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96000 msgr2=0x55d2e5bf1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96000 0x55d2e5bf1080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55d2e6794fc0 tx=0x55d2e5c58c60 comp rx=0 tx=0).stop 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 msgr2=0x55d2e5bf0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55d2e66d9ec0 tx=0x55d2e59f9980 comp rx=0 tx=0).stop 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 shutdown_connections 2024-04-27T04:34:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d2e5c96000 0x55d2e5bf1080 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 --2- 172.21.15.191:0/1666893853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e5908800 0x55d2e5bf0580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:42.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 >> 172.21.15.191:0/1666893853 conn(0x55d2e5908400 msgr2=0x55d2e59f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:42.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 shutdown_connections 2024-04-27T04:34:42.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:42.932+0000 7ff2f356ba80 1 -- 172.21.15.191:0/1666893853 wait complete. 2024-04-27T04:34:42.964 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:42.964 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [], 2024-04-27T04:34:42.965 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [] 2024-04-27T04:34:42.965 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:42.965 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-27T04:34:42.965 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:42.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:42.966 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.048+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.048+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1b292a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565026ef8c00 0x5650271e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1b292a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x565027223500 con 0x565026ef8800 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1b292a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x565026f9aa00 con 0x565026ef8c00 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e18a3b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565026ef8c00 0x5650271e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1923c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x565026ef8800 0x565026fe4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55256/0 (socket says 172.21.15.191:55256) 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1923c700 1 -- 172.21.15.191:0/993302649 learned_addr learned my addr 172.21.15.191:0/993302649 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e18a3b700 1 -- 172.21.15.191:0/993302649 >> v1:172.21.15.138:6789/0 conn(0x565026ef8800 legacy=0x565026fe4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e18a3b700 1 -- 172.21.15.191:0/993302649 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f21d40 con 0x565026ef8c00 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1823a700 1 -- 172.21.15.191:0/993302649 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219443295 0 0) 0x565027cc6000 con 0x565026ef8800 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/993302649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8c00 0x5650271e0580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5650271db890 tx=0x565026fe9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7dd59f80b3c0da server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1823a700 1 -- 172.21.15.191:0/993302649 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb14a0 con 0x565026ef8c00 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1823a700 1 -- 172.21.15.191:0/993302649 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565027223800 con 0x565026ef8c00 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.049+0000 7f1e1823a700 1 -- 172.21.15.191:0/993302649 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb1600 con 0x565026ef8c00 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 -- 172.21.15.191:0/993302649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8c00 msgr2=0x5650271e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/993302649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8c00 0x5650271e0580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5650271db890 tx=0x565026fe9920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 -- 172.21.15.191:0/993302649 shutdown_connections 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/993302649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8c00 0x5650271e0580 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 -- 172.21.15.191:0/993302649 >> 172.21.15.191:0/993302649 conn(0x565026ef8400 msgr2=0x565026fe4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 -- 172.21.15.191:0/993302649 shutdown_connections 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.050+0000 7f1e1b292a80 1 -- 172.21.15.191:0/993302649 wait complete. 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1b292a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1b292a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565026f9a500 con 0x565026ef8800 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1923c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1923c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38304/0 (socket says 172.21.15.191:38304) 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1923c700 1 -- 172.21.15.191:0/1767736507 learned_addr learned my addr 172.21.15.191:0/1767736507 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1923c700 1 -- 172.21.15.191:0/1767736507 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f21ba0 con 0x565026ef8800 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e1923c700 1 --2- 172.21.15.191:0/1767736507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x565027cc9770 tx=0x565026fe9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bedbfa07bca4d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e17a39700 1 -- 172.21.15.191:0/1767736507 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb14a0 con 0x565026ef8800 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e17a39700 1 -- 172.21.15.191:0/1767736507 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565027223b00 con 0x565026ef8800 2024-04-27T04:34:43.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.051+0000 7f1e17a39700 1 -- 172.21.15.191:0/1767736507 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb1600 con 0x565026ef8800 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1767736507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 msgr2=0x5650271e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1767736507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x565027cc9770 tx=0x565026fe9b60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1767736507 shutdown_connections 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1767736507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1767736507 >> 172.21.15.191:0/1767736507 conn(0x565026ef8400 msgr2=0x565026fe5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1767736507 shutdown_connections 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1767736507 wait complete. 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.052+0000 7f1e1b292a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565026f9a3c0 con 0x565026ef8800 2024-04-27T04:34:43.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1923c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1923c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38316/0 (socket says 172.21.15.191:38316) 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1923c700 1 -- 172.21.15.191:0/846163702 learned_addr learned my addr 172.21.15.191:0/846163702 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1923c700 1 -- 172.21.15.191:0/846163702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f21a00 con 0x565026ef8800 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1923c700 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x565027cc9e30 tx=0x565026fe9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e16236700 1 -- 172.21.15.191:0/846163702 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb18c0 con 0x565026ef8800 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e16236700 1 -- 172.21.15.191:0/846163702 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565026f99c80 con 0x565026ef8800 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565026f21860 con 0x565026ef8800 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e16236700 1 -- 172.21.15.191:0/846163702 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb14a0 con 0x565026ef8800 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.053+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565026f216c0 con 0x565026ef8800 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e16236700 1 -- 172.21.15.191:0/846163702 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565027c9ab00 con 0x565026ef8800 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e1b292a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e1b292a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565026f9a000 con 0x565026ef9400 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e16236700 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288000 0x5650271e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e16236700 1 -- 172.21.15.191:0/846163702 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x565026f73180 con 0x565026ef8800 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e19a3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e19a3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38326/0 (socket says 172.21.15.191:38326) 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e19a3d700 1 -- 172.21.15.191:0/4083280119 learned_addr learned my addr 172.21.15.191:0/4083280119 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e1923c700 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288000 0x5650271e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e19a3d700 1 -- 172.21.15.191:0/4083280119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f20000 con 0x565026ef9400 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e1923c700 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288000 0x5650271e1080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x565027d8af00 tx=0x565027244240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/4083280119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x565027d8a360 tx=0x5650272450e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b99434087b8222a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e13a31700 1 -- 172.21.15.191:0/4083280119 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb18c0 con 0x565026ef9400 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e13a31700 1 -- 172.21.15.191:0/4083280119 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565027222300 con 0x565026ef9400 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.054+0000 7f1e13a31700 1 -- 172.21.15.191:0/4083280119 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb14a0 con 0x565026ef9400 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- 172.21.15.191:0/4083280119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 msgr2=0x5650271e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/4083280119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x565027d8a360 tx=0x5650272450e0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- 172.21.15.191:0/4083280119 shutdown_connections 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/4083280119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- 172.21.15.191:0/4083280119 >> 172.21.15.191:0/4083280119 conn(0x565026ef8c00 msgr2=0x565026fe5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- 172.21.15.191:0/4083280119 shutdown_connections 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- 172.21.15.191:0/4083280119 wait complete. 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.055+0000 7f1e1b292a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565026f34640 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e19a3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e19a3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38334/0 (socket says 172.21.15.191:38334) 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 learned_addr learned my addr 172.21.15.191:0/1278802440 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f20ea0 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x565027d8bf20 tx=0x565027244d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e1222e700 1 -- 172.21.15.191:0/1278802440 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb1b80 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e1222e700 1 -- 172.21.15.191:0/1278802440 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565026f99c80 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565026f20820 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e1222e700 1 -- 172.21.15.191:0/1278802440 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb18c0 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.056+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565026f209c0 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e1222e700 1 -- 172.21.15.191:0/1278802440 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565027c9adc0 con 0x565026ef9400 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e1222e700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288c00 0x5650271e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e1222e700 1 -- 172.21.15.191:0/1278802440 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x565026f73340 con 0x565026ef9400 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288c00 0x5650271e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565026ef9800 0x5650271e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565026ef9c00 con 0x565026ef9800 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565026ef9800 0x5650271e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.057+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288c00 0x5650271e1600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5650271db890 tx=0x565026fe9800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565026ef9800 0x5650271e1b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 realm 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289000 con 0x565026ef9800 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289400 con 0x565026ef9800 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.058+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289800 con 0x565026ef9800 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565027289c00 0x5650271e2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289000 con 0x565027289c00 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1923c700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565027289c00 0x5650271e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1923c700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565027289c00 0x5650271e2100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 20 searching for the correct realm 2024-04-27T04:34:43.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.059+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289400 con 0x565027289c00 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e1923c700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565027289400 0x5650271e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289000 con 0x565027289400 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565027289400 0x5650271e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565027289400 0x5650271e2c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x565027289000 0x5650271e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.060+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.061+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x565027289000 0x5650271e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.061+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x565027289000 0x5650271e3180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.061+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.061+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.061+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.061+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289400 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.062+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289400 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.063+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.064+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.065+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289400 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.066+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289400 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.067+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.067+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289c00 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.067+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.067+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.067+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.067+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289400 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565026ef9c00 con 0x565026ef9800 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350400 con 0x565026ef9800 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.068+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350800 con 0x565027289c00 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 zone default found 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 4 Realm: () 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.106 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 zonegroup default 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350c00 con 0x565027289000 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351000 con 0x565027289c00 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351400 con 0x565027289c00 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 started sync module instance, tier type = 2024-04-27T04:34:43.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.069+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351800 con 0x565027289400 2024-04-27T04:34:43.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.070+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.070+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94902253615872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350800 con 0x565027289400 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.070+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351000 con 0x565026ef9800 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.073+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.073+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94902253617024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351400 con 0x565026ef9800 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.073+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94902253615872] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.073+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350400 con 0x565027289c00 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.074+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94902253617024] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.074+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.074+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94902253618176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350c00 con 0x565027289c00 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.074+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565026ef9c00 con 0x565027289400 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.075+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94902253618176] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.076+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.076+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94902253619328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027289800 con 0x565027289400 2024-04-27T04:34:43.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.076+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565026ef9800 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.078+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.078+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94902253620480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351800 con 0x565026ef9800 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.078+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350800 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.081+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94902253619328] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.081+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.081+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94902253621632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351c00 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.081+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351000 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.084+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94902253620480] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.084+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.084+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94902253622784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351400 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.084+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350000 con 0x565027289000 2024-04-27T04:34:43.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(58 notify.7 [create] v31'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94902253621632] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94902253623936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351800 con 0x565027289000 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=0 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=1 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=2 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=3 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=4 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=5 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94902253622784] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.087+0000 7f1e1b292a80 20 add_watcher() i=6 2024-04-27T04:34:43.123 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-27T04:34:43.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.089+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94902253623936] v31'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.089+0000 7f1e1b292a80 20 add_watcher() i=7 2024-04-27T04:34:43.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.089+0000 7f1e1b292a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.089+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350800 con 0x565026ef9800 2024-04-27T04:34:43.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.090+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.090+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94902255206400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351c00 con 0x565026ef9800 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94902255206400] v31'54 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e0020a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e1b292a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e1b292a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e1b292a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.091+0000 7f1e1b292a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565026f34780 con 0x565027351400 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1e18a3b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1e18a3b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38338/0 (socket says 172.21.15.191:38338) 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1e18a3b700 1 -- 172.21.15.191:0/2889060862 learned_addr learned my addr 172.21.15.191:0/2889060862 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1e18a3b700 1 -- 172.21.15.191:0/2889060862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f20b60 con 0x565027351400 2024-04-27T04:34:43.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/2889060862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5650272f1800 tx=0x565026fe9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb55b65203167da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1dffa09700 1 -- 172.21.15.191:0/2889060862 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565026fb1760 con 0x565027351400 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1dffa09700 1 -- 172.21.15.191:0/2889060862 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565027370000 con 0x565027351400 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.092+0000 7f1dffa09700 1 -- 172.21.15.191:0/2889060862 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565027372000 con 0x565027351400 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 -- 172.21.15.191:0/2889060862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 msgr2=0x5650271e2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/2889060862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5650272f1800 tx=0x565026fe9980 comp rx=0 tx=0).stop 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 -- 172.21.15.191:0/2889060862 shutdown_connections 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/2889060862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 -- 172.21.15.191:0/2889060862 >> 172.21.15.191:0/2889060862 conn(0x565027351000 msgr2=0x565026fe6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 -- 172.21.15.191:0/2889060862 shutdown_connections 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 -- 172.21.15.191:0/2889060862 wait complete. 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 Processor -- start 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 -- start start 2024-04-27T04:34:43.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.093+0000 7f1e1b292a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e1b292a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565026f9a000 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e18a3b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e18a3b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38354/0 (socket says 172.21.15.191:38354) 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e18a3b700 1 -- 172.21.15.191:0/3660641288 learned_addr learned my addr 172.21.15.191:0/3660641288 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e18a3b700 1 -- 172.21.15.191:0/3660641288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565026f20d00 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e18a3b700 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x565027365260 tx=0x565026fe98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1dfe206700 1 -- 172.21.15.191:0/3660641288 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565027372160 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1dfe206700 1 -- 172.21.15.191:0/3660641288 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565027370180 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565026f20340 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1dfe206700 1 -- 172.21.15.191:0/3660641288 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650273722c0 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.094+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565026f21d40 con 0x565027351400 2024-04-27T04:34:43.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.095+0000 7f1dfe206700 1 -- 172.21.15.191:0/3660641288 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565027c9bb80 con 0x565027351400 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.095+0000 7f1dfe206700 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5650273ec000 0x5650271e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.095+0000 7f1dfe206700 1 -- 172.21.15.191:0/3660641288 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x565026f73500 con 0x565027351400 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.095+0000 7f1e1b292a80 5 note: GC not initialized 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.095+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5650273ec000 0x5650271e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.096+0000 7f1e19a3d700 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5650273ec000 0x5650271e3700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5650272f1470 tx=0x565027373320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.096+0000 7f1dfba01700 20 reqs_thread_entry: start 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.096+0000 7f1e1b292a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.096+0000 7f1e1b292a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.096+0000 7f1df99fd700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:13 2024 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.096+0000 7f1df91db700 20 reqs_thread_entry: start 2024-04-27T04:34:43.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.097+0000 7f1e1b292a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.097+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027351800 con 0x565026ef9800 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.097+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:could not fetch user info: 2024-04-27T04:34:43.097+0000 7f1e1b292a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:no user info saved 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.098+0000 7f1e0020a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.098+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94902255206400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565027350800 con 0x565026ef9800 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.099+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94902255206400] v31'55 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.099+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94902253615872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ec400 con 0x565027289400 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.101+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94902253615872] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.101+0000 7f1e1b292a80 20 remove_watcher() i=0 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.101+0000 7f1e1b292a80 2 removed watcher, disabling cache 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.101+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94902253617024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ec800 con 0x565026ef9800 2024-04-27T04:34:43.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.102+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94902253617024] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.102+0000 7f1e1b292a80 20 remove_watcher() i=1 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.102+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94902253618176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ecc00 con 0x565027289c00 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.104+0000 7f1e1923c700 1 -- 172.21.15.191:0/1278802440 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94902253618176] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345200 con 0x565027289c00 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.104+0000 7f1e1b292a80 20 remove_watcher() i=2 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.104+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94902253619328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ed000 con 0x565027289400 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.105+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94902253619328] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027345440 con 0x565027289400 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.105+0000 7f1e1b292a80 20 remove_watcher() i=3 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.105+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94902253620480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ed400 con 0x565026ef9800 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.107+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94902253620480] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565026ef9800 2024-04-27T04:34:43.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.107+0000 7f1e1b292a80 20 remove_watcher() i=4 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.107+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94902253621632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ed800 con 0x565027289000 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.108+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94902253621632] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.108+0000 7f1e1b292a80 20 remove_watcher() i=5 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.108+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94902253622784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273edc00 con 0x565027289000 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.109+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94902253622784] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.109+0000 7f1e1b292a80 20 remove_watcher() i=6 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.109+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94902253623936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5650273ec800 con 0x565027289000 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e19a3d700 1 -- 172.21.15.191:0/1278802440 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94902253623936] v31'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565027344b40 con 0x565027289000 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 20 remove_watcher() i=7 2024-04-27T04:34:43.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x565027289000 msgr2=0x5650271e3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x565027289000 0x5650271e3180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565026ef9800 msgr2=0x5650271e1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565026ef9800 0x5650271e1b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565027289400 msgr2=0x5650271e2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565027289400 0x5650271e2c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565027289c00 msgr2=0x5650271e2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565027289c00 0x5650271e2100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288c00 msgr2=0x5650271e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288c00 0x5650271e1600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5650271db890 tx=0x565026fe9800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 msgr2=0x5650271e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/1278802440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef9400 0x5650271e0b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x565027d8bf20 tx=0x565027244d20 comp rx=0 tx=0).stop 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 reap_dead start 2024-04-27T04:34:43.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e18a3b700 1 -- 172.21.15.191:0/1278802440 reap_dead start 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 shutdown_connections 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 >> 172.21.15.191:0/1278802440 conn(0x565026ef8c00 msgr2=0x565026fe5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 shutdown_connections 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/1278802440 wait complete. 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5650273ec000 msgr2=0x5650271e3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5650273ec000 0x5650271e3700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5650272f1470 tx=0x565027373320 comp rx=0 tx=0).stop 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 msgr2=0x5650271e2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x565027365260 tx=0x565026fe98c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 shutdown_connections 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5650273ec000 0x5650271e3700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/3660641288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565027351400 0x5650271e2680 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.110+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 >> 172.21.15.191:0/3660641288 conn(0x565027351000 msgr2=0x565026fe6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 shutdown_connections 2024-04-27T04:34:43.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/3660641288 wait complete. 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288000 msgr2=0x5650271e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288000 0x5650271e1080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x565027d8af00 tx=0x565027244240 comp rx=0 tx=0).stop 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 msgr2=0x5650271e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x565027cc9e30 tx=0x565026fe9a40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 shutdown_connections 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565027288000 0x5650271e1080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 --2- 172.21.15.191:0/846163702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565026ef8800 0x5650271e0580 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 >> 172.21.15.191:0/846163702 conn(0x565026ef8400 msgr2=0x565026fe5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 shutdown_connections 2024-04-27T04:34:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.111+0000 7f1e1b292a80 1 -- 172.21.15.191:0/846163702 wait complete. 2024-04-27T04:34:43.133 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-27T04:34:43.134 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-27T04:34:43.134 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-04-27T04:34:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.187+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f7660694a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f7660694a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561ff1c17500 con 0x561ff18ecc00 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f7660694a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x561ff198ea00 con 0x561ff18ec800 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f765e63e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f765de3d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x561ff18ecc00 0x561ff19d8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55268/0 (socket says 172.21.15.191:55268) 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f765de3d700 1 -- 172.21.15.191:0/426826218 learned_addr learned my addr 172.21.15.191:0/426826218 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f765e63e700 1 -- 172.21.15.191:0/426826218 >> v1:172.21.15.138:6789/0 conn(0x561ff18ecc00 legacy=0x561ff19d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f765e63e700 1 -- 172.21.15.191:0/426826218 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1915d40 con 0x561ff18ec800 2024-04-27T04:34:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.188+0000 7f765e63e700 1 --2- 172.21.15.191:0/426826218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x561ff1bcf9e0 tx=0x561ff19dd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7661608c0ddf80 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.189+0000 7f765d63c700 1 -- 172.21.15.191:0/426826218 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a54a0 con 0x561ff18ec800 2024-04-27T04:34:43.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.189+0000 7f765d63c700 1 -- 172.21.15.191:0/426826218 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff1c17800 con 0x561ff18ec800 2024-04-27T04:34:43.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.189+0000 7f765d63c700 1 -- 172.21.15.191:0/426826218 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a5600 con 0x561ff18ec800 2024-04-27T04:34:43.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 -- 172.21.15.191:0/426826218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 msgr2=0x561ff1bd4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 --2- 172.21.15.191:0/426826218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x561ff1bcf9e0 tx=0x561ff19dd920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 -- 172.21.15.191:0/426826218 shutdown_connections 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 --2- 172.21.15.191:0/426826218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 -- 172.21.15.191:0/426826218 >> 172.21.15.191:0/426826218 conn(0x561ff18ec400 msgr2=0x561ff19d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 -- 172.21.15.191:0/426826218 shutdown_connections 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 -- 172.21.15.191:0/426826218 wait complete. 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.190+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f7660694a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f7660694a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff198e500 con 0x561ff18ec800 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765e63e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765e63e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38372/0 (socket says 172.21.15.191:38372) 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765e63e700 1 -- 172.21.15.191:0/2910072517 learned_addr learned my addr 172.21.15.191:0/2910072517 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765e63e700 1 -- 172.21.15.191:0/2910072517 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1915ba0 con 0x561ff18ec800 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765e63e700 1 --2- 172.21.15.191:0/2910072517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x561ff26b3d40 tx=0x561ff19dd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaef4fa05eefe904 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765ce3b700 1 -- 172.21.15.191:0/2910072517 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a5760 con 0x561ff18ec800 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765ce3b700 1 -- 172.21.15.191:0/2910072517 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff1c17b00 con 0x561ff18ec800 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.191+0000 7f765ce3b700 1 -- 172.21.15.191:0/2910072517 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a54a0 con 0x561ff18ec800 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- 172.21.15.191:0/2910072517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 msgr2=0x561ff1bd4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 --2- 172.21.15.191:0/2910072517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x561ff26b3d40 tx=0x561ff19dd800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- 172.21.15.191:0/2910072517 shutdown_connections 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 --2- 172.21.15.191:0/2910072517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- 172.21.15.191:0/2910072517 >> 172.21.15.191:0/2910072517 conn(0x561ff18ec400 msgr2=0x561ff19d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- 172.21.15.191:0/2910072517 shutdown_connections 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- 172.21.15.191:0/2910072517 wait complete. 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f7660694a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff198e3c0 con 0x561ff18ec800 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f765e63e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f765e63e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38374/0 (socket says 172.21.15.191:38374) 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.192+0000 7f765e63e700 1 -- 172.21.15.191:0/2663356039 learned_addr learned my addr 172.21.15.191:0/2663356039 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f765e63e700 1 -- 172.21.15.191:0/2663356039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1915a00 con 0x561ff18ec800 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f765e63e700 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x561ff26b3620 tx=0x561ff19dd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f765b638700 1 -- 172.21.15.191:0/2663356039 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a5600 con 0x561ff18ec800 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f765b638700 1 -- 172.21.15.191:0/2663356039 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff1c17c80 con 0x561ff18ec800 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f765b638700 1 -- 172.21.15.191:0/2663356039 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a58c0 con 0x561ff18ec800 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ff1915860 con 0x561ff18ec800 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ff19156c0 con 0x561ff18ec800 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.193+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765b638700 1 -- 172.21.15.191:0/2663356039 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561ff268eb00 con 0x561ff18ec800 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f7660694a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f7660694a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff1928640 con 0x561ff18ed400 2024-04-27T04:34:43.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765b638700 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74000 0x561ff1bd5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765ee3f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765b638700 1 -- 172.21.15.191:0/2663356039 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x561ff1967180 con 0x561ff18ec800 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765ee3f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38376/0 (socket says 172.21.15.191:38376) 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765ee3f700 1 -- 172.21.15.191:0/3033598236 learned_addr learned my addr 172.21.15.191:0/3033598236 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765e63e700 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74000 0x561ff1bd5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765ee3f700 1 -- 172.21.15.191:0/3033598236 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1914000 con 0x561ff18ed400 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765ee3f700 1 --2- 172.21.15.191:0/3033598236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x561ff26b3e90 tx=0x561ff1c3cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d880584843cd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f765e63e700 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74000 0x561ff1bd5080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x561ff2764ff0 tx=0x561ff1c3cc60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f7658e33700 1 -- 172.21.15.191:0/3033598236 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a5600 con 0x561ff18ed400 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f7658e33700 1 -- 172.21.15.191:0/3033598236 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff198dc80 con 0x561ff18ed400 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.194+0000 7f7658e33700 1 -- 172.21.15.191:0/3033598236 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a58c0 con 0x561ff18ed400 2024-04-27T04:34:43.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- 172.21.15.191:0/3033598236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 msgr2=0x561ff1bd4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 --2- 172.21.15.191:0/3033598236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x561ff26b3e90 tx=0x561ff1c3cb40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- 172.21.15.191:0/3033598236 shutdown_connections 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 --2- 172.21.15.191:0/3033598236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- 172.21.15.191:0/3033598236 >> 172.21.15.191:0/3033598236 conn(0x561ff18ecc00 msgr2=0x561ff19d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- 172.21.15.191:0/3033598236 shutdown_connections 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- 172.21.15.191:0/3033598236 wait complete. 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f7660694a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff19288c0 con 0x561ff18ed400 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f765ee3f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f765ee3f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38392/0 (socket says 172.21.15.191:38392) 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.195+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 learned_addr learned my addr 172.21.15.191:0/25455851 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1914ea0 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f765ee3f700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x561ff2764000 tx=0x561ff19dd8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f7657630700 1 -- 172.21.15.191:0/25455851 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a5760 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ff1914820 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f7657630700 1 -- 172.21.15.191:0/25455851 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff198db00 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f7657630700 1 -- 172.21.15.191:0/25455851 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff19a5600 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.196+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ff19149c0 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f7657630700 1 -- 172.21.15.191:0/25455851 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561ff268edc0 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f7657630700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74c00 0x561ff1bd5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f7657630700 1 -- 172.21.15.191:0/25455851 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x561ff1967340 con 0x561ff18ed400 2024-04-27T04:34:43.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f765de3d700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74c00 0x561ff1bd5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561ff18ed800 0x561ff1bd5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff18edc00 con 0x561ff18ed800 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f765ee3f700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561ff18ed800 0x561ff1bd5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.197+0000 7f765de3d700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74c00 0x561ff1bd5600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x561ff1c77e30 tx=0x561ff1d2c0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f765ee3f700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561ff18ed800 0x561ff1bd5b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 realm 2024-04-27T04:34:43.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75000 con 0x561ff18ed800 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75400 con 0x561ff18ed800 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75800 con 0x561ff18ed800 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.198+0000 7f7660694a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561ff1c75c00 0x561ff1bd6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75000 con 0x561ff1c75c00 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f765e63e700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561ff1c75c00 0x561ff1bd6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f765e63e700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561ff1c75c00 0x561ff1bd6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f7660694a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f7660694a80 20 searching for the correct realm 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.199+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75400 con 0x561ff1c75c00 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765e63e700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561ff1c75400 0x561ff1bd6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75000 con 0x561ff1c75400 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765de3d700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561ff1c75400 0x561ff1bd6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765de3d700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561ff1c75400 0x561ff1bd6c00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765de3d700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561ff1c75000 0x561ff1bd7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.200+0000 7f765ee3f700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561ff1c75000 0x561ff1bd7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.201+0000 7f765ee3f700 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561ff1c75000 0x561ff1bd7180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.201+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.201+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75400 2024-04-27T04:34:43.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.202+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.203+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75400 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.204+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.205+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75400 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.206+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.207+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.207+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.207+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.207+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.207+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.207+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75400 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75c00 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.208+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75400 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff18edc00 con 0x561ff18ed800 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42400 con 0x561ff18ed800 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42800 con 0x561ff1c75c00 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 zone default found 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 4 Realm: () 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 zonegroup default 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.209+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42c00 con 0x561ff1c75000 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43000 con 0x561ff1c75c00 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43400 con 0x561ff1c75c00 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 started sync module instance, tier type = 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.210+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43800 con 0x561ff1c75400 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.211+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.211+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94695199652608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42800 con 0x561ff1c75400 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.211+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43000 con 0x561ff18ed800 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.212+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.212+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94695199653760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43400 con 0x561ff18ed800 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.212+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42400 con 0x561ff1c75c00 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.214+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94695199653760] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.214+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94695199652608] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.217+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.217+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94695199654912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42c00 con 0x561ff1c75c00 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.217+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff18edc00 con 0x561ff1c75400 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.217+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.218+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94695199656064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1c75800 con 0x561ff1c75400 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.218+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff18ed800 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.218+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94695199654912] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.220+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.220+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94695199657216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43800 con 0x561ff18ed800 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.220+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42800 con 0x561ff1c75000 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.223+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.223+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94695199658368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43c00 con 0x561ff1c75000 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.223+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43000 con 0x561ff1c75000 2024-04-27T04:34:43.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.223+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94695199656064] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.225+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94695199657216] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.226+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94695199658368] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.228+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.228+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94695199659520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43400 con 0x561ff1c75000 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.228+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42000 con 0x561ff1c75000 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94695199659520] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94695199660672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43800 con 0x561ff1c75000 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=0 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=1 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=2 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=3 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=4 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=5 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.229+0000 7f7660694a80 20 add_watcher() i=6 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.230+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94695199660672] v31'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.230+0000 7f7660694a80 20 add_watcher() i=7 2024-04-27T04:34:43.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.230+0000 7f7660694a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.230+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42800 con 0x561ff18ed800 2024-04-27T04:34:43.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.231+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.231+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94695201234944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43c00 con 0x561ff18ed800 2024-04-27T04:34:43.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94695201234944] v31'56 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f764560c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f7660694a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f7660694a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f7660694a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.233+0000 7f7660694a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff198e3c0 con 0x561ff1d43400 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f765de3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f765de3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38404/0 (socket says 172.21.15.191:38404) 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f765de3d700 1 -- 172.21.15.191:0/2067966263 learned_addr learned my addr 172.21.15.191:0/2067966263 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f765de3d700 1 -- 172.21.15.191:0/2067966263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1914b60 con 0x561ff1d43400 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f765de3d700 1 --2- 172.21.15.191:0/2067966263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x561ff1cd95f0 tx=0x561ff1c3df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a05d448643413 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f7644e0b700 1 -- 172.21.15.191:0/2067966263 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff1d62000 con 0x561ff1d43400 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f7644e0b700 1 -- 172.21.15.191:0/2067966263 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff1d6a000 con 0x561ff1d43400 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.234+0000 7f7644e0b700 1 -- 172.21.15.191:0/2067966263 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff1d62160 con 0x561ff1d43400 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- 172.21.15.191:0/2067966263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 msgr2=0x561ff1bd6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 --2- 172.21.15.191:0/2067966263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x561ff1cd95f0 tx=0x561ff1c3df80 comp rx=0 tx=0).stop 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- 172.21.15.191:0/2067966263 shutdown_connections 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 --2- 172.21.15.191:0/2067966263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- 172.21.15.191:0/2067966263 >> 172.21.15.191:0/2067966263 conn(0x561ff1d43000 msgr2=0x561ff19da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- 172.21.15.191:0/2067966263 shutdown_connections 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- 172.21.15.191:0/2067966263 wait complete. 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 Processor -- start 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- start start 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.235+0000 7f7660694a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff198e500 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f765de3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f765de3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38420/0 (socket says 172.21.15.191:38420) 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f765de3d700 1 -- 172.21.15.191:0/232989065 learned_addr learned my addr 172.21.15.191:0/232989065 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f765de3d700 1 -- 172.21.15.191:0/232989065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff1914d00 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f765de3d700 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x561ff1d52720 tx=0x561ff1d2c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f7643608700 1 -- 172.21.15.191:0/232989065 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff1d622c0 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f7643608700 1 -- 172.21.15.191:0/232989065 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ff1d6a180 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f7643608700 1 -- 172.21.15.191:0/232989065 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561ff1d62000 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ff1914340 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.236+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ff1914b60 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.237+0000 7f7643608700 1 -- 172.21.15.191:0/232989065 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561ff268fb80 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.237+0000 7f7643608700 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1de4000 0x561ff1bd7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.237+0000 7f7643608700 1 -- 172.21.15.191:0/232989065 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x561ff1967500 con 0x561ff1d43400 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.237+0000 7f765ee3f700 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1de4000 0x561ff1bd7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.237+0000 7f7660694a80 5 note: GC not initialized 2024-04-27T04:34:43.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f765ee3f700 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1de4000 0x561ff1bd7700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x561ff1cd99e0 tx=0x561ff1d2c360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f7640e03700 20 reqs_thread_entry: start 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f7660694a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f7660694a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f763edff700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:13 2024 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f763e5dd700 20 reqs_thread_entry: start 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f7660694a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.238+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43800 con 0x561ff18ed800 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42800 con 0x561ff18ed800 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 197+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.239+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de4400 con 0x561ff1c75000 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f7660694a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f7660694a80 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f7660694a80 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f7660694a80 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f7660694a80 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.240+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de4800 con 0x561ff18ed800 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~357] v31'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7660694a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7660694a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7660694a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94695201237248 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de4c00 con 0x561ff18ed800 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== watch-notify(notify (1) cookie 94695199657216 notify 133143986176 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x561ff1ce1d40 con 0x561ff18ed800 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7658e33700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986176 cookie 94695199657216 notifier 4292 bl.length()=533 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7658e33700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7658e33700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(66 notify.4 [notify cookie 94695201237248 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7658e33700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1e54000 con 0x561ff18ed800 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== watch-notify(notify_complete (2) cookie 94695201237248 notify 133143986176 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561ff1dafd40 con 0x561ff18ed800 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.241+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de5000 con 0x561ff18ed800 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 239+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7660694a80 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7660694a80 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7660694a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94695201237248 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de5400 con 0x561ff1c75400 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== watch-notify(notify (1) cookie 94695199656064 notify 133143986176 ret 0) v3 ==== 211+0+0 (crc 0 0 0) 0x561ff1ce1ba0 con 0x561ff1c75400 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7658e33700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986176 cookie 94695199656064 notifier 4292 bl.length()=169 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(69 notify.3 [notify cookie 94695201237248 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7658e33700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7658e33700 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T04:34:43.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f7658e33700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1e54400 con 0x561ff1c75400 2024-04-27T04:34:43.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.243+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.244+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== watch-notify(notify_complete (2) cookie 94695201237248 notify 133143986176 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561ff1ce01a0 con 0x561ff1c75400 2024-04-27T04:34:43.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.244+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de5800 con 0x561ff1c75000 2024-04-27T04:34:43.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7660694a80 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-27T04:34:43.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7660694a80 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T04:34:43.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7660694a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-27T04:34:43.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94695201237248 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de5c00 con 0x561ff1c75400 2024-04-27T04:34:43.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== watch-notify(notify (1) cookie 94695199656064 notify 133143986177 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x561ff1ce01a0 con 0x561ff1c75400 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7658e33700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986177 cookie 94695199656064 notifier 4292 bl.length()=177 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(72 notify.3 [notify cookie 94695201237248 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7658e33700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7658e33700 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f7658e33700 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d43c00 con 0x561ff1c75400 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== watch-notify(notify_complete (2) cookie 94695201237248 notify 133143986177 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561ff1ce0340 con 0x561ff1c75400 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.245+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:43.252 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:43.253 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.254 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.255 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.255 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:43.255 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:43.255 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:43.255 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:43.255 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.246+0000 7f764560c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.246+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94695201234944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de5400 con 0x561ff18ed800 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.247+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94695201234944] v31'57 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.247+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94695199652608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de5c00 con 0x561ff1c75400 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.248+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94695199652608] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.248+0000 7f7660694a80 20 remove_watcher() i=0 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.248+0000 7f7660694a80 2 removed watcher, disabling cache 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.248+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94695199653760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de4400 con 0x561ff18ed800 2024-04-27T04:34:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.250+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94695199653760] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.250+0000 7f7660694a80 20 remove_watcher() i=1 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.250+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94695199654912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de4800 con 0x561ff1c75c00 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.251+0000 7f765e63e700 1 -- 172.21.15.191:0/25455851 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94695199654912] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35200 con 0x561ff1c75c00 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.251+0000 7f7660694a80 20 remove_watcher() i=2 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.251+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94695199656064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1de4c00 con 0x561ff1c75400 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.252+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94695199656064] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d35440 con 0x561ff1c75400 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.252+0000 7f7660694a80 20 remove_watcher() i=3 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.252+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94695199657216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1d42800 con 0x561ff18ed800 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.253+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94695199657216] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff18ed800 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.253+0000 7f7660694a80 20 remove_watcher() i=4 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.253+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94695199658368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1ca2000 con 0x561ff1c75000 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.254+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94695199658368] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.254+0000 7f7660694a80 20 remove_watcher() i=5 2024-04-27T04:34:43.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.254+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94695199659520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1ca2400 con 0x561ff1c75000 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.255+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94695199659520] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.255+0000 7f7660694a80 20 remove_watcher() i=6 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.255+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94695199660672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561ff1ca2800 con 0x561ff1c75000 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f765ee3f700 1 -- 172.21.15.191:0/25455851 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94695199660672] v31'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561ff1d34b40 con 0x561ff1c75000 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 20 remove_watcher() i=7 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561ff1c75000 msgr2=0x561ff1bd7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561ff1c75000 0x561ff1bd7180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561ff18ed800 msgr2=0x561ff1bd5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561ff18ed800 0x561ff1bd5b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561ff1c75400 msgr2=0x561ff1bd6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561ff1c75400 0x561ff1bd6c00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561ff1c75c00 msgr2=0x561ff1bd6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.256+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561ff1c75c00 0x561ff1bd6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74c00 msgr2=0x561ff1bd5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74c00 0x561ff1bd5600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x561ff1c77e30 tx=0x561ff1d2c0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 msgr2=0x561ff1bd4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/25455851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ed400 0x561ff1bd4b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x561ff2764000 tx=0x561ff19dd8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 reap_dead start 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f765de3d700 1 -- 172.21.15.191:0/25455851 reap_dead start 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 shutdown_connections 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 >> 172.21.15.191:0/25455851 conn(0x561ff18ecc00 msgr2=0x561ff19d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 shutdown_connections 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/25455851 wait complete. 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1de4000 msgr2=0x561ff1bd7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1de4000 0x561ff1bd7700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x561ff1cd99e0 tx=0x561ff1d2c360 comp rx=0 tx=0).stop 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 msgr2=0x561ff1bd6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x561ff1d52720 tx=0x561ff1d2c4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 shutdown_connections 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1de4000 0x561ff1bd7700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/232989065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff1d43400 0x561ff1bd6680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 >> 172.21.15.191:0/232989065 conn(0x561ff1d43000 msgr2=0x561ff19da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 shutdown_connections 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/232989065 wait complete. 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74000 msgr2=0x561ff1bd5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74000 0x561ff1bd5080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x561ff2764ff0 tx=0x561ff1c3cc60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 msgr2=0x561ff1bd4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.257+0000 7f7660694a80 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x561ff26b3620 tx=0x561ff19dd920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.258+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 shutdown_connections 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.258+0000 7f7660694a80 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561ff1c74000 0x561ff1bd5080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.258+0000 7f7660694a80 1 --2- 172.21.15.191:0/2663356039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ff18ec800 0x561ff1bd4580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.258+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 >> 172.21.15.191:0/2663356039 conn(0x561ff18ec400 msgr2=0x561ff19d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.258+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 shutdown_connections 2024-04-27T04:34:43.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.258+0000 7f7660694a80 1 -- 172.21.15.191:0/2663356039 wait complete. 2024-04-27T04:34:43.266 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:43.266 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-27T04:34:43.266 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-27T04:34:43.266 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-04-27T04:34:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f2786a03a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b3de288c00 0x55b3de570580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f2786a03a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b3de5b3500 con 0x55b3de288800 2024-04-27T04:34:43.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f2786a03a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b3de32aa00 con 0x55b3de288c00 2024-04-27T04:34:43.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f27849ad700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55b3de288800 0x55b3de374800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55276/0 (socket says 172.21.15.191:55276) 2024-04-27T04:34:43.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f27841ac700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b3de288c00 0x55b3de570580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f27849ad700 1 -- 172.21.15.191:0/2338800341 learned_addr learned my addr 172.21.15.191:0/2338800341 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f27841ac700 1 -- 172.21.15.191:0/2338800341 >> v1:172.21.15.138:6789/0 conn(0x55b3de288800 legacy=0x55b3de374800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f27841ac700 1 -- 172.21.15.191:0/2338800341 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b1d40 con 0x55b3de288c00 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.315+0000 7f27839ab700 1 -- 172.21.15.191:0/2338800341 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154147717 0 0) 0x55b3df056000 con 0x55b3de288800 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f27841ac700 1 --2- 172.21.15.191:0/2338800341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288c00 0x55b3de570580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55b3de56b890 tx=0x55b3de379920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aa22cf7f020a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f27839ab700 1 -- 172.21.15.191:0/2338800341 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3414a0 con 0x55b3de288c00 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f27839ab700 1 -- 172.21.15.191:0/2338800341 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de5b3800 con 0x55b3de288c00 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f27839ab700 1 -- 172.21.15.191:0/2338800341 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de341600 con 0x55b3de288c00 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 -- 172.21.15.191:0/2338800341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288c00 msgr2=0x55b3de570580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2338800341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288c00 0x55b3de570580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55b3de56b890 tx=0x55b3de379920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 -- 172.21.15.191:0/2338800341 shutdown_connections 2024-04-27T04:34:43.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2338800341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288c00 0x55b3de570580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 -- 172.21.15.191:0/2338800341 >> 172.21.15.191:0/2338800341 conn(0x55b3de288400 msgr2=0x55b3de374000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 -- 172.21.15.191:0/2338800341 shutdown_connections 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.316+0000 7f2786a03a80 1 -- 172.21.15.191:0/2338800341 wait complete. 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f2786a03a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f2786a03a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3de32a500 con 0x55b3de288800 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38430/0 (socket says 172.21.15.191:38430) 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f27849ad700 1 -- 172.21.15.191:0/2870339328 learned_addr learned my addr 172.21.15.191:0/2870339328 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.317+0000 7f27849ad700 1 -- 172.21.15.191:0/2870339328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b1ba0 con 0x55b3de288800 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f27849ad700 1 --2- 172.21.15.191:0/2870339328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55b3df059770 tx=0x55b3de379b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a850640eca731187 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f27831aa700 1 -- 172.21.15.191:0/2870339328 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3411e0 con 0x55b3de288800 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f27831aa700 1 -- 172.21.15.191:0/2870339328 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de5b3b00 con 0x55b3de288800 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f27831aa700 1 -- 172.21.15.191:0/2870339328 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3414a0 con 0x55b3de288800 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 -- 172.21.15.191:0/2870339328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 msgr2=0x55b3de570580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2870339328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55b3df059770 tx=0x55b3de379b60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 -- 172.21.15.191:0/2870339328 shutdown_connections 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2870339328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 -- 172.21.15.191:0/2870339328 >> 172.21.15.191:0/2870339328 conn(0x55b3de288400 msgr2=0x55b3de375000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 -- 172.21.15.191:0/2870339328 shutdown_connections 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 -- 172.21.15.191:0/2870339328 wait complete. 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.318+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f2786a03a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f2786a03a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3de32a3c0 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38442/0 (socket says 172.21.15.191:38442) 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27849ad700 1 -- 172.21.15.191:0/2644206310 learned_addr learned my addr 172.21.15.191:0/2644206310 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27849ad700 1 -- 172.21.15.191:0/2644206310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b1a00 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27849ad700 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55b3df059e30 tx=0x55b3de379a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27819a7700 1 -- 172.21.15.191:0/2644206310 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de341600 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27819a7700 1 -- 172.21.15.191:0/2644206310 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de329c80 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b3de2b1860 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f27819a7700 1 -- 172.21.15.191:0/2644206310 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3411e0 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.319+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3de2b16c0 con 0x55b3de288800 2024-04-27T04:34:43.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27819a7700 1 -- 172.21.15.191:0/2644206310 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b3df02ab00 con 0x55b3de288800 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27819a7700 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de289400 0x55b3de571080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27819a7700 1 -- 172.21.15.191:0/2644206310 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b3de303180 con 0x55b3de288800 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f2786a03a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f2786a03a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3de2c4640 con 0x55b3de289800 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27851ae700 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de289400 0x55b3de571080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38456/0 (socket says 172.21.15.191:38456) 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.320+0000 7f27849ad700 1 -- 172.21.15.191:0/3996448706 learned_addr learned my addr 172.21.15.191:0/3996448706 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f27851ae700 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de289400 0x55b3de571080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b3df059f20 tx=0x55b3de5d8de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f27849ad700 1 -- 172.21.15.191:0/3996448706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b0000 con 0x55b3de289800 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f27849ad700 1 --2- 172.21.15.191:0/3996448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55b3df118330 tx=0x55b3de5d8240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364ee0570f9a1927 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f277f1a2700 1 -- 172.21.15.191:0/3996448706 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3414a0 con 0x55b3de289800 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f277f1a2700 1 -- 172.21.15.191:0/3996448706 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de5b2300 con 0x55b3de289800 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f277f1a2700 1 -- 172.21.15.191:0/3996448706 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3418c0 con 0x55b3de289800 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f2786a03a80 1 -- 172.21.15.191:0/3996448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 msgr2=0x55b3de570b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3996448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55b3df118330 tx=0x55b3de5d8240 comp rx=0 tx=0).stop 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f2786a03a80 1 -- 172.21.15.191:0/3996448706 shutdown_connections 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3996448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.321+0000 7f2786a03a80 1 -- 172.21.15.191:0/3996448706 >> 172.21.15.191:0/3996448706 conn(0x55b3de288c00 msgr2=0x55b3de375800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f2786a03a80 1 -- 172.21.15.191:0/3996448706 shutdown_connections 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f2786a03a80 1 -- 172.21.15.191:0/3996448706 wait complete. 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f2786a03a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f2786a03a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3de2c48c0 con 0x55b3de289800 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f27849ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38462/0 (socket says 172.21.15.191:38462) 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 learned_addr learned my addr 172.21.15.191:0/3482881279 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.322+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b0ea0 con 0x55b3de289800 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f27849ad700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55b3de630c30 tx=0x55b3de5d8fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f277d99f700 1 -- 172.21.15.191:0/3482881279 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3414a0 con 0x55b3de289800 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f277d99f700 1 -- 172.21.15.191:0/3482881279 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de329e00 con 0x55b3de289800 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f277d99f700 1 -- 172.21.15.191:0/3482881279 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de3418c0 con 0x55b3de289800 2024-04-27T04:34:43.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b3de2b0820 con 0x55b3de289800 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3de2b09c0 con 0x55b3de289800 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.323+0000 7f277d99f700 1 -- 172.21.15.191:0/3482881279 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b3df02adc0 con 0x55b3de289800 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f277d99f700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de634c00 0x55b3de571600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f277d99f700 1 -- 172.21.15.191:0/3482881279 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b3de303340 con 0x55b3de289800 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f27841ac700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de634c00 0x55b3de571600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b3de289c00 0x55b3de571b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635000 con 0x55b3de289c00 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f27851ae700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b3de289c00 0x55b3de571b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f27841ac700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de634c00 0x55b3de571600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b3de56b890 tx=0x55b3de379800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.324+0000 7f27851ae700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b3de289c00 0x55b3de571b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 realm 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635400 con 0x55b3de289c00 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635800 con 0x55b3de289c00 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635c00 con 0x55b3de289c00 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b3de635000 0x55b3de572100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635400 con 0x55b3de635000 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.325+0000 7f27849ad700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b3de635000 0x55b3de572100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f27849ad700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b3de635000 0x55b3de572100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f2786a03a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f2786a03a80 20 searching for the correct realm 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6000 con 0x55b3de635000 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f27849ad700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b3de6b6000 0x55b3de572c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635400 con 0x55b3de6b6000 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.326+0000 7f27841ac700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b3de6b6000 0x55b3de572c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.327+0000 7f27841ac700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b3de6b6000 0x55b3de572c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.327+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.327+0000 7f27841ac700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b3de635400 0x55b3de573180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.327+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.327+0000 7f27851ae700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b3de635400 0x55b3de573180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.327+0000 7f27851ae700 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b3de635400 0x55b3de573180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.328+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de6b6000 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.329+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de6b6000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.330+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.331+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de6b6000 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.332+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de6b6000 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.333+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635400 2024-04-27T04:34:43.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de289c00 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de6b6000 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6400 con 0x55b3de289c00 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.334+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6800 con 0x55b3de289c00 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6c00 con 0x55b3de635000 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 zone default found 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 4 Realm: () 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 zonegroup default 2024-04-27T04:34:43.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7000 con 0x55b3de635400 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.335+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7400 con 0x55b3de635000 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6400 con 0x55b3de635000 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 20 started sync module instance, tier type = 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.336+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6800 con 0x55b3de6b6000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.337+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.337+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94231017716480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7000 con 0x55b3de6b6000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.337+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7c00 con 0x55b3de289c00 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.339+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94231017716480] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.339+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.339+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94231017717632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635c00 con 0x55b3de289c00 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.340+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de635800 con 0x55b3de635000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.342+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.342+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94231017718784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7800 con 0x55b3de635000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.342+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6800 con 0x55b3de6b6000 2024-04-27T04:34:43.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.342+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94231017717632] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.345+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94231017718784] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.345+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.345+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94231017719936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7000 con 0x55b3de6b6000 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.345+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6c00 con 0x55b3de289c00 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.348+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.348+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94231017721088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7400 con 0x55b3de289c00 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.348+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6400 con 0x55b3de635400 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.348+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94231017719936] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.349+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94231017721088] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.349+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.349+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94231017722240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de634400 con 0x55b3de635400 2024-04-27T04:34:43.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.349+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6be000 con 0x55b3de635400 2024-04-27T04:34:43.392 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-04-27T04:34:43.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.354+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94231017722240] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.354+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.354+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94231017723392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6be400 con 0x55b3de635400 2024-04-27T04:34:43.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.354+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6be800 con 0x55b3de635400 2024-04-27T04:34:43.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94231017723392] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94231017724544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6bec00 con 0x55b3de635400 2024-04-27T04:34:43.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=0 2024-04-27T04:34:43.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=1 2024-04-27T04:34:43.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=2 2024-04-27T04:34:43.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=3 2024-04-27T04:34:43.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=4 2024-04-27T04:34:43.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=5 2024-04-27T04:34:43.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.355+0000 7f2786a03a80 20 add_watcher() i=6 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.356+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94231017724544] v31'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.356+0000 7f2786a03a80 20 add_watcher() i=7 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.356+0000 7f2786a03a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.356+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6bf000 con 0x55b3de289c00 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.356+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.356+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94231019151360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6bf400 con 0x55b3de289c00 2024-04-27T04:34:43.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.358+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94231019151360] v31'58 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.358+0000 7f276b97b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.358+0000 7f2786a03a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.358+0000 7f2786a03a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f2786a03a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f2786a03a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3de32a3c0 con 0x55b3de6bfc00 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f27841ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f27841ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38468/0 (socket says 172.21.15.191:38468) 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f27841ac700 1 -- 172.21.15.191:0/3070453219 learned_addr learned my addr 172.21.15.191:0/3070453219 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f27841ac700 1 -- 172.21.15.191:0/3070453219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b0b60 con 0x55b3de6bfc00 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f27841ac700 1 --2- 172.21.15.191:0/3070453219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55b3de651680 tx=0x55b3de379980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fb5d1f235379b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.359+0000 7f276b17a700 1 -- 172.21.15.191:0/3070453219 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de341760 con 0x55b3de6bfc00 2024-04-27T04:34:43.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f276b17a700 1 -- 172.21.15.191:0/3070453219 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de6d0000 con 0x55b3de6bfc00 2024-04-27T04:34:43.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f276b17a700 1 -- 172.21.15.191:0/3070453219 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de6c4160 con 0x55b3de6bfc00 2024-04-27T04:34:43.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 -- 172.21.15.191:0/3070453219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 msgr2=0x55b3de572680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3070453219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55b3de651680 tx=0x55b3de379980 comp rx=0 tx=0).stop 2024-04-27T04:34:43.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 -- 172.21.15.191:0/3070453219 shutdown_connections 2024-04-27T04:34:43.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3070453219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 -- 172.21.15.191:0/3070453219 >> 172.21.15.191:0/3070453219 conn(0x55b3de6bf800 msgr2=0x55b3de376000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 -- 172.21.15.191:0/3070453219 shutdown_connections 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.360+0000 7f2786a03a80 1 -- 172.21.15.191:0/3070453219 wait complete. 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2786a03a80 1 Processor -- start 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2786a03a80 1 -- start start 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2786a03a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2786a03a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3de32a500 con 0x55b3de6bfc00 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f27841ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f27841ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38474/0 (socket says 172.21.15.191:38474) 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f27841ac700 1 -- 172.21.15.191:0/1583955515 learned_addr learned my addr 172.21.15.191:0/1583955515 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f27841ac700 1 -- 172.21.15.191:0/1583955515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3de2b0d00 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f27841ac700 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55b3de6cb260 tx=0x55b3de3798c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2769977700 1 -- 172.21.15.191:0/1583955515 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de6c42c0 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2769977700 1 -- 172.21.15.191:0/1583955515 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3de6d0180 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b3de2b0340 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2769977700 1 -- 172.21.15.191:0/1583955515 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b3de6c4420 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.361+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3de2b1d40 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.362+0000 7f2769977700 1 -- 172.21.15.191:0/1583955515 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b3df02bb80 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.362+0000 7f2769977700 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de6b7c00 0x55b3de573700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.362+0000 7f2769977700 1 -- 172.21.15.191:0/1583955515 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b3de3036c0 con 0x55b3de6bfc00 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.362+0000 7f2786a03a80 5 note: GC not initialized 2024-04-27T04:34:43.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.362+0000 7f27851ae700 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de6b7c00 0x55b3de573700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.363+0000 7f27851ae700 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de6b7c00 0x55b3de573700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55b3de651d40 tx=0x55b3de6e72c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.363+0000 7f2767172700 20 reqs_thread_entry: start 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.363+0000 7f2786a03a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.363+0000 7f2786a03a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.363+0000 7f276516e700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:13 2024 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f276494c700 20 reqs_thread_entry: start 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de634400 con 0x55b3de635400 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T04:34:43.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6c00 con 0x55b3de289c00 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+23 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 20 rados_obj.operate() r=0 bl.length=7 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.364+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b7400 con 0x55b3de289c00 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 foo [call out=48b,read 0~357 out=357b] v0'0 uv1 ondisk = 0) v8 ==== 189+0+405 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f2786a03a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f2786a03a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x11 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f2786a03a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f2786a03a80 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f276b97b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.365+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94231019151360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6b6400 con 0x55b3de289c00 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.366+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94231019151360] v31'59 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.366+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94231017716480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de634400 con 0x55b3de6b6000 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.367+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94231017716480] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.367+0000 7f2786a03a80 20 remove_watcher() i=0 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.367+0000 7f2786a03a80 2 removed watcher, disabling cache 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.367+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94231017717632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6be000 con 0x55b3de289c00 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.368+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94231017717632] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.368+0000 7f2786a03a80 20 remove_watcher() i=1 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.368+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94231017718784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6be400 con 0x55b3de635000 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.369+0000 7f27849ad700 1 -- 172.21.15.191:0/3482881279 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94231017718784] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af200 con 0x55b3de635000 2024-04-27T04:34:43.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.369+0000 7f2786a03a80 20 remove_watcher() i=2 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.369+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94231017719936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6be800 con 0x55b3de6b6000 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.370+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94231017719936] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6af440 con 0x55b3de6b6000 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.370+0000 7f2786a03a80 20 remove_watcher() i=3 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.370+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94231017721088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6bec00 con 0x55b3de289c00 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.371+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94231017721088] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de289c00 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.371+0000 7f2786a03a80 20 remove_watcher() i=4 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.371+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94231017722240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6bf000 con 0x55b3de635400 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.372+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94231017722240] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.372+0000 7f2786a03a80 20 remove_watcher() i=5 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.372+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94231017723392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6bf400 con 0x55b3de635400 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.373+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94231017723392] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.373+0000 7f2786a03a80 20 remove_watcher() i=6 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.373+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94231017724544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b3de6a4000 con 0x55b3de635400 2024-04-27T04:34:43.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f27851ae700 1 -- 172.21.15.191:0/3482881279 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94231017724544] v31'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b3de6aeb40 con 0x55b3de635400 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 20 remove_watcher() i=7 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b3de635400 msgr2=0x55b3de573180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b3de635400 0x55b3de573180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b3de289c00 msgr2=0x55b3de571b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b3de289c00 0x55b3de571b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b3de6b6000 msgr2=0x55b3de572c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b3de6b6000 0x55b3de572c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b3de635000 msgr2=0x55b3de572100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b3de635000 0x55b3de572100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de634c00 msgr2=0x55b3de571600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de634c00 0x55b3de571600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b3de56b890 tx=0x55b3de379800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 msgr2=0x55b3de570b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 --2- 172.21.15.191:0/3482881279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de289800 0x55b3de570b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55b3de630c30 tx=0x55b3de5d8fc0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 reap_dead start 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f27841ac700 1 -- 172.21.15.191:0/3482881279 reap_dead start 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 shutdown_connections 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 >> 172.21.15.191:0/3482881279 conn(0x55b3de288c00 msgr2=0x55b3de375800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 shutdown_connections 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.374+0000 7f2786a03a80 1 -- 172.21.15.191:0/3482881279 wait complete. 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de6b7c00 msgr2=0x55b3de573700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de6b7c00 0x55b3de573700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55b3de651d40 tx=0x55b3de6e72c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 msgr2=0x55b3de572680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55b3de6cb260 tx=0x55b3de3798c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 shutdown_connections 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de6b7c00 0x55b3de573700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/1583955515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de6bfc00 0x55b3de572680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 >> 172.21.15.191:0/1583955515 conn(0x55b3de6bf800 msgr2=0x55b3de376000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 shutdown_connections 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/1583955515 wait complete. 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de289400 msgr2=0x55b3de571080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de289400 0x55b3de571080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b3df059f20 tx=0x55b3de5d8de0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 msgr2=0x55b3de570580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55b3df059e30 tx=0x55b3de379a40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 shutdown_connections 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b3de289400 0x55b3de571080 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 --2- 172.21.15.191:0/2644206310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3de288800 0x55b3de570580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 >> 172.21.15.191:0/2644206310 conn(0x55b3de288400 msgr2=0x55b3de375000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 shutdown_connections 2024-04-27T04:34:43.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.375+0000 7f2786a03a80 1 -- 172.21.15.191:0/2644206310 wait complete. 2024-04-27T04:34:43.408 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:43.409 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:43.409 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:43.454 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.455 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.453+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259ebaea80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b086f9cc00 0x55b087284580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259ebaea80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b0872c7500 con 0x55b086f9c800 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259ebaea80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b08703ea00 con 0x55b086f9cc00 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259c357700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b086f9cc00 0x55b087284580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259cb58700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55b086f9c800 0x55b087088800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55280/0 (socket says 172.21.15.191:55280) 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259cb58700 1 -- 172.21.15.191:0/1048479702 learned_addr learned my addr 172.21.15.191:0/1048479702 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259c357700 1 -- 172.21.15.191:0/1048479702 >> v1:172.21.15.138:6789/0 conn(0x55b086f9c800 legacy=0x55b087088800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259c357700 1 -- 172.21.15.191:0/1048479702 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc5d40 con 0x55b086f9cc00 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259bb56700 1 -- 172.21.15.191:0/1048479702 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695207214 0 0) 0x55b087d6a000 con 0x55b086f9c800 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.454+0000 7f259c357700 1 --2- 172.21.15.191:0/1048479702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9cc00 0x55b087284580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55b08727f890 tx=0x55b08708d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9216f7407b2bdb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259bb56700 1 -- 172.21.15.191:0/1048479702 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870554a0 con 0x55b086f9cc00 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259bb56700 1 -- 172.21.15.191:0/1048479702 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0872c7800 con 0x55b086f9cc00 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259bb56700 1 -- 172.21.15.191:0/1048479702 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b087055600 con 0x55b086f9cc00 2024-04-27T04:34:43.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 -- 172.21.15.191:0/1048479702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9cc00 msgr2=0x55b087284580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1048479702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9cc00 0x55b087284580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55b08727f890 tx=0x55b08708d920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 -- 172.21.15.191:0/1048479702 shutdown_connections 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1048479702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9cc00 0x55b087284580 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 -- 172.21.15.191:0/1048479702 >> 172.21.15.191:0/1048479702 conn(0x55b086f9c400 msgr2=0x55b087088000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 -- 172.21.15.191:0/1048479702 shutdown_connections 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.455+0000 7f259ebaea80 1 -- 172.21.15.191:0/1048479702 wait complete. 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259ebaea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259ebaea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08703e500 con 0x55b086f9c800 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259cb58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259cb58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38488/0 (socket says 172.21.15.191:38488) 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259cb58700 1 -- 172.21.15.191:0/1240577194 learned_addr learned my addr 172.21.15.191:0/1240577194 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.456+0000 7f259cb58700 1 -- 172.21.15.191:0/1240577194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc5ba0 con 0x55b086f9c800 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259cb58700 1 --2- 172.21.15.191:0/1240577194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55b087d6d770 tx=0x55b08708db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c0b1699a7da287 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259b355700 1 -- 172.21.15.191:0/1240577194 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870554a0 con 0x55b086f9c800 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259b355700 1 -- 172.21.15.191:0/1240577194 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0872c7b00 con 0x55b086f9c800 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259b355700 1 -- 172.21.15.191:0/1240577194 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b087055600 con 0x55b086f9c800 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 -- 172.21.15.191:0/1240577194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 msgr2=0x55b087284580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1240577194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55b087d6d770 tx=0x55b08708db60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 -- 172.21.15.191:0/1240577194 shutdown_connections 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1240577194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 -- 172.21.15.191:0/1240577194 >> 172.21.15.191:0/1240577194 conn(0x55b086f9c400 msgr2=0x55b087089000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 -- 172.21.15.191:0/1240577194 shutdown_connections 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 -- 172.21.15.191:0/1240577194 wait complete. 2024-04-27T04:34:43.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.457+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259ebaea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259ebaea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08703e3c0 con 0x55b086f9c800 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259cb58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259cb58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38500/0 (socket says 172.21.15.191:38500) 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259cb58700 1 -- 172.21.15.191:0/1817948656 learned_addr learned my addr 172.21.15.191:0/1817948656 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259cb58700 1 -- 172.21.15.191:0/1817948656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc5a00 con 0x55b086f9c800 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259cb58700 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55b087d6de30 tx=0x55b08708da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f2599b52700 1 -- 172.21.15.191:0/1817948656 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870558c0 con 0x55b086f9c800 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f2599b52700 1 -- 172.21.15.191:0/1817948656 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b08703dc80 con 0x55b086f9c800 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b086fc5860 con 0x55b086f9c800 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f2599b52700 1 -- 172.21.15.191:0/1817948656 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870554a0 con 0x55b086f9c800 2024-04-27T04:34:43.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.458+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b086fc56c0 con 0x55b086f9c800 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f2599b52700 1 -- 172.21.15.191:0/1817948656 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b087d3eb00 con 0x55b086f9c800 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259ebaea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259ebaea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08703e000 con 0x55b086f9d400 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f2599b52700 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732e000 0x55b087285080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f2599b52700 1 -- 172.21.15.191:0/1817948656 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b087017180 con 0x55b086f9c800 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259d359700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259d359700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38502/0 (socket says 172.21.15.191:38502) 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259cb58700 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732e000 0x55b087285080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.459+0000 7f259d359700 1 -- 172.21.15.191:0/3568409962 learned_addr learned my addr 172.21.15.191:0/3568409962 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259d359700 1 -- 172.21.15.191:0/3568409962 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc4000 con 0x55b086f9d400 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259cb58700 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732e000 0x55b087285080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55b087e2ef00 tx=0x55b0872ea240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259d359700 1 --2- 172.21.15.191:0/3568409962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55b087d6def0 tx=0x55b0872eb0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a040f33c0f40b4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259734d700 1 -- 172.21.15.191:0/3568409962 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870558c0 con 0x55b086f9d400 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259734d700 1 -- 172.21.15.191:0/3568409962 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0872c6300 con 0x55b086f9d400 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259734d700 1 -- 172.21.15.191:0/3568409962 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870554a0 con 0x55b086f9d400 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 -- 172.21.15.191:0/3568409962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 msgr2=0x55b087284b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 --2- 172.21.15.191:0/3568409962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55b087d6def0 tx=0x55b0872eb0e0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 -- 172.21.15.191:0/3568409962 shutdown_connections 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 --2- 172.21.15.191:0/3568409962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 -- 172.21.15.191:0/3568409962 >> 172.21.15.191:0/3568409962 conn(0x55b086f9cc00 msgr2=0x55b087089800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 -- 172.21.15.191:0/3568409962 shutdown_connections 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.460+0000 7f259ebaea80 1 -- 172.21.15.191:0/3568409962 wait complete. 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259ebaea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259ebaea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b086fd8640 con 0x55b086f9d400 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259d359700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259d359700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38518/0 (socket says 172.21.15.191:38518) 2024-04-27T04:34:43.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 learned_addr learned my addr 172.21.15.191:0/4046992959 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc4ea0 con 0x55b086f9d400 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259d359700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55b087e2e450 tx=0x55b08708df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f2595b4a700 1 -- 172.21.15.191:0/4046992959 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b087055b80 con 0x55b086f9d400 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f2595b4a700 1 -- 172.21.15.191:0/4046992959 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b08703dc80 con 0x55b086f9d400 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b086fc4820 con 0x55b086f9d400 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f2595b4a700 1 -- 172.21.15.191:0/4046992959 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0870558c0 con 0x55b086f9d400 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.461+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b086fc49c0 con 0x55b086f9d400 2024-04-27T04:34:43.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f2595b4a700 1 -- 172.21.15.191:0/4046992959 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b087d3edc0 con 0x55b086f9d400 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f2595b4a700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732ec00 0x55b087285600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f2595b4a700 1 -- 172.21.15.191:0/4046992959 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b087017340 con 0x55b086f9d400 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f259c357700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732ec00 0x55b087285600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b086f9d800 0x55b087285b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b086f9dc00 con 0x55b086f9d800 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.462+0000 7f259d359700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b086f9d800 0x55b087285b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259c357700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732ec00 0x55b087285600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b08727f890 tx=0x55b08708d800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259d359700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b086f9d800 0x55b087285b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 20 realm 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f000 con 0x55b086f9d800 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f400 con 0x55b086f9d800 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.463+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f800 con 0x55b086f9d800 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b08732fc00 0x55b087286100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f000 con 0x55b08732fc00 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259cb58700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b08732fc00 0x55b087286100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259cb58700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b08732fc00 0x55b087286100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 20 searching for the correct realm 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.464+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f400 con 0x55b08732fc00 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259cb58700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b08732f400 0x55b087286c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f000 con 0x55b08732f400 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259c357700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b08732f400 0x55b087286c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259c357700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b08732f400 0x55b087286c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259c357700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b08732f000 0x55b087287180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.465+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.466+0000 7f259d359700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b08732f000 0x55b087287180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.466+0000 7f259d359700 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b08732f000 0x55b087287180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.466+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.466+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.466+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.466+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f400 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.467+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f400 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.468+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.469+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f400 2024-04-27T04:34:43.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.470+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.471+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f400 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732fc00 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f400 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.472+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b086f9dc00 con 0x55b086f9d800 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2400 con 0x55b086f9d800 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2800 con 0x55b08732fc00 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 zone default found 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 4 Realm: () 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 zonegroup default 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.473+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2c00 con 0x55b08732f000 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3000 con 0x55b08732fc00 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3400 con 0x55b08732fc00 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 started sync module instance, tier type = 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.474+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3800 con 0x55b08732f400 2024-04-27T04:34:43.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.475+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.475+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94216670132992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2800 con 0x55b08732f400 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.475+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3000 con 0x55b086f9d800 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.476+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94216670132992] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.476+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.476+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94216670134144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3400 con 0x55b086f9d800 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.476+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2400 con 0x55b08732fc00 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.477+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.477+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94216670135296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2c00 con 0x55b08732fc00 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.477+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b086f9dc00 con 0x55b08732f400 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.477+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94216670134144] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.478+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94216670135296] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.478+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.478+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94216670136448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08732f800 con 0x55b08732f400 2024-04-27T04:34:43.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.478+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b086f9d800 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.479+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.479+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94216670137600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3800 con 0x55b086f9d800 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.479+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2800 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.479+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94216670136448] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.480+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.480+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94216670137600] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.480+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94216670138752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3c00 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.480+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3000 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.481+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.481+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94216670139904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3400 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.481+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2000 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94216670138752] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.528 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94216670141056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3800 con 0x55b08732f000 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 20 add_watcher() i=0 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 20 add_watcher() i=1 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 20 add_watcher() i=2 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 20 add_watcher() i=3 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 20 add_watcher() i=4 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.482+0000 7f259ebaea80 20 add_watcher() i=5 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94216670139904] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259ebaea80 20 add_watcher() i=6 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94216670141056] v31'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259ebaea80 20 add_watcher() i=7 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259ebaea80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2800 con 0x55b086f9d800 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.529 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.483+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94216671715328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3c00 con 0x55b086f9d800 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94216671715328] v31'60 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f2583b26700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259ebaea80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259ebaea80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259ebaea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259ebaea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b086fd8780 con 0x55b0873f3400 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.485+0000 7f259c357700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f259c357700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38532/0 (socket says 172.21.15.191:38532) 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f259c357700 1 -- 172.21.15.191:0/710371106 learned_addr learned my addr 172.21.15.191:0/710371106 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f259c357700 1 -- 172.21.15.191:0/710371106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc4b60 con 0x55b0873f3400 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f259c357700 1 --2- 172.21.15.191:0/710371106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55b087397800 tx=0x55b08708d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15edfdb71fd1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.530 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f2583325700 1 -- 172.21.15.191:0/710371106 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b087055760 con 0x55b0873f3400 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f2583325700 1 -- 172.21.15.191:0/710371106 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b087412000 con 0x55b0873f3400 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.486+0000 7f2583325700 1 -- 172.21.15.191:0/710371106 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b087414000 con 0x55b0873f3400 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- 172.21.15.191:0/710371106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 msgr2=0x55b087286680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 --2- 172.21.15.191:0/710371106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55b087397800 tx=0x55b08708d980 comp rx=0 tx=0).stop 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- 172.21.15.191:0/710371106 shutdown_connections 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 --2- 172.21.15.191:0/710371106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- 172.21.15.191:0/710371106 >> 172.21.15.191:0/710371106 conn(0x55b0873f3000 msgr2=0x55b08708a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- 172.21.15.191:0/710371106 shutdown_connections 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- 172.21.15.191:0/710371106 wait complete. 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 Processor -- start 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- start start 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259ebaea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08703e000 con 0x55b0873f3400 2024-04-27T04:34:43.531 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259c357700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259c357700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38548/0 (socket says 172.21.15.191:38548) 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.487+0000 7f259c357700 1 -- 172.21.15.191:0/843972705 learned_addr learned my addr 172.21.15.191:0/843972705 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f259c357700 1 -- 172.21.15.191:0/843972705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b086fc4d00 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f259c357700 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55b087407260 tx=0x55b08708d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f2581b22700 1 -- 172.21.15.191:0/843972705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b087414160 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f2581b22700 1 -- 172.21.15.191:0/843972705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b087412180 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f2581b22700 1 -- 172.21.15.191:0/843972705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0874142c0 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b086fc4340 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.488+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b086fc5d40 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.489+0000 7f2581b22700 1 -- 172.21.15.191:0/843972705 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b087d3fb80 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.489+0000 7f2581b22700 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08748e000 0x55b087287700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.489+0000 7f2581b22700 1 -- 172.21.15.191:0/843972705 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b087017500 con 0x55b0873f3400 2024-04-27T04:34:43.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.489+0000 7f259d359700 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08748e000 0x55b087287700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.489+0000 7f259ebaea80 5 note: GC not initialized 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.489+0000 7f259d359700 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08748e000 0x55b087287700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b087397350 tx=0x55b087415200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f257f31d700 20 reqs_thread_entry: start 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f259ebaea80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f259ebaea80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f257d319700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:13 2024 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f257caf7700 20 reqs_thread_entry: start 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f259ebaea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.490+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f3800 con 0x55b086f9d800 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.491+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.491+0000 7f259ebaea80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:43.534 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.535 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:43.536 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.491+0000 7f2583b26700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.491+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94216671715328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0873f2800 con 0x55b086f9d800 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.492+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94216671715328] v31'61 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.492+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94216670132992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748e400 con 0x55b08732f400 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.493+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94216670132992] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.493+0000 7f259ebaea80 20 remove_watcher() i=0 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.493+0000 7f259ebaea80 2 removed watcher, disabling cache 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.493+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94216670134144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748e800 con 0x55b086f9d800 2024-04-27T04:34:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.495+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94216670134144] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.495+0000 7f259ebaea80 20 remove_watcher() i=1 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.495+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94216670135296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748ec00 con 0x55b08732fc00 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.496+0000 7f259cb58700 1 -- 172.21.15.191:0/4046992959 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94216670135296] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7200 con 0x55b08732fc00 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.496+0000 7f259ebaea80 20 remove_watcher() i=2 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.496+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94216670136448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748f000 con 0x55b08732f400 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.496+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94216670136448] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e7440 con 0x55b08732f400 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.496+0000 7f259ebaea80 20 remove_watcher() i=3 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.496+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94216670137600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748f400 con 0x55b086f9d800 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.497+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94216670137600] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b086f9d800 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.497+0000 7f259ebaea80 20 remove_watcher() i=4 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.497+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94216670138752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748f800 con 0x55b08732f000 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.498+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94216670138752] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.498+0000 7f259ebaea80 20 remove_watcher() i=5 2024-04-27T04:34:43.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.498+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94216670139904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748fc00 con 0x55b08732f000 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.499+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94216670139904] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.499+0000 7f259ebaea80 20 remove_watcher() i=6 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.499+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94216670141056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b08748e800 con 0x55b08732f000 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259d359700 1 -- 172.21.15.191:0/4046992959 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94216670141056] v31'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0873e6b40 con 0x55b08732f000 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 20 remove_watcher() i=7 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b08732f000 msgr2=0x55b087287180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b08732f000 0x55b087287180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b086f9d800 msgr2=0x55b087285b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b086f9d800 0x55b087285b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b08732f400 msgr2=0x55b087286c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b08732f400 0x55b087286c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b08732fc00 msgr2=0x55b087286100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b08732fc00 0x55b087286100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732ec00 msgr2=0x55b087285600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732ec00 0x55b087285600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b08727f890 tx=0x55b08708d800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 msgr2=0x55b087284b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 --2- 172.21.15.191:0/4046992959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9d400 0x55b087284b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55b087e2e450 tx=0x55b08708df20 comp rx=0 tx=0).stop 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 reap_dead start 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259c357700 1 -- 172.21.15.191:0/4046992959 reap_dead start 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 shutdown_connections 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 >> 172.21.15.191:0/4046992959 conn(0x55b086f9cc00 msgr2=0x55b087089800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 shutdown_connections 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.500+0000 7f259ebaea80 1 -- 172.21.15.191:0/4046992959 wait complete. 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08748e000 msgr2=0x55b087287700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08748e000 0x55b087287700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b087397350 tx=0x55b087415200 comp rx=0 tx=0).stop 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 msgr2=0x55b087286680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55b087407260 tx=0x55b08708d8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 shutdown_connections 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08748e000 0x55b087287700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/843972705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0873f3400 0x55b087286680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 >> 172.21.15.191:0/843972705 conn(0x55b0873f3000 msgr2=0x55b08708a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 shutdown_connections 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/843972705 wait complete. 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732e000 msgr2=0x55b087285080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732e000 0x55b087285080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55b087e2ef00 tx=0x55b0872ea240 comp rx=0 tx=0).stop 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 msgr2=0x55b087284580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55b087d6de30 tx=0x55b08708da40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 shutdown_connections 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b08732e000 0x55b087285080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 --2- 172.21.15.191:0/1817948656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b086f9c800 0x55b087284580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 >> 172.21.15.191:0/1817948656 conn(0x55b086f9c400 msgr2=0x55b087089000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 shutdown_connections 2024-04-27T04:34:43.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.501+0000 7f259ebaea80 1 -- 172.21.15.191:0/1817948656 wait complete. 2024-04-27T04:34:43.547 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:43.547 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-27T04:34:43.547 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-27T04:34:43.547 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-27T04:34:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.593+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.593+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.593+0000 7f5921a93a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55699aeb4c00 0x55699b19c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f5921a93a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55699b1df500 con 0x55699aeb4800 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f5921a93a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55699af56a00 con 0x55699aeb4c00 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591f23c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55699aeb4c00 0x55699b19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591fa3d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55699aeb4800 0x55699afa0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55292/0 (socket says 172.21.15.191:55292) 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591fa3d700 1 -- 172.21.15.191:0/28083425 learned_addr learned my addr 172.21.15.191:0/28083425 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591ea3b700 1 -- 172.21.15.191:0/28083425 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125587977 0 0) 0x55699bc82000 con 0x55699aeb4800 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591ea3b700 1 -- 172.21.15.191:0/28083425 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55699b1df680 con 0x55699aeb4800 2024-04-27T04:34:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591f23c700 1 -- 172.21.15.191:0/28083425 >> v1:172.21.15.138:6789/0 conn(0x55699aeb4800 legacy=0x55699afa0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591f23c700 1 -- 172.21.15.191:0/28083425 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aeddd40 con 0x55699aeb4c00 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591f23c700 1 --2- 172.21.15.191:0/28083425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4c00 0x55699b19c580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55699b197890 tx=0x55699afa5980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a49cc9fdd0b62de server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591ea3b700 1 -- 172.21.15.191:0/28083425 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d4a0 con 0x55699aeb4c00 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591ea3b700 1 -- 172.21.15.191:0/28083425 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699b1df500 con 0x55699aeb4c00 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.594+0000 7f591ea3b700 1 -- 172.21.15.191:0/28083425 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d600 con 0x55699aeb4c00 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 -- 172.21.15.191:0/28083425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4c00 msgr2=0x55699b19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 --2- 172.21.15.191:0/28083425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4c00 0x55699b19c580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55699b197890 tx=0x55699afa5980 comp rx=0 tx=0).stop 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 -- 172.21.15.191:0/28083425 shutdown_connections 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 --2- 172.21.15.191:0/28083425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4c00 0x55699b19c580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 -- 172.21.15.191:0/28083425 >> 172.21.15.191:0/28083425 conn(0x55699aeb4400 msgr2=0x55699afa0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 -- 172.21.15.191:0/28083425 shutdown_connections 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.595+0000 7f5921a93a80 1 -- 172.21.15.191:0/28083425 wait complete. 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f5921a93a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f5921a93a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55699af56500 con 0x55699aeb4800 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591fa3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591fa3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38572/0 (socket says 172.21.15.191:38572) 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591fa3d700 1 -- 172.21.15.191:0/634754345 learned_addr learned my addr 172.21.15.191:0/634754345 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591fa3d700 1 -- 172.21.15.191:0/634754345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aeddba0 con 0x55699aeb4800 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591fa3d700 1 --2- 172.21.15.191:0/634754345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55699bc85770 tx=0x55699afa5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd299d44f9aa04e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591e23a700 1 -- 172.21.15.191:0/634754345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d4a0 con 0x55699aeb4800 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591e23a700 1 -- 172.21.15.191:0/634754345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699b1dfb00 con 0x55699aeb4800 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.596+0000 7f591e23a700 1 -- 172.21.15.191:0/634754345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d600 con 0x55699aeb4800 2024-04-27T04:34:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- 172.21.15.191:0/634754345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 msgr2=0x55699b19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 --2- 172.21.15.191:0/634754345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55699bc85770 tx=0x55699afa5b60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- 172.21.15.191:0/634754345 shutdown_connections 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 --2- 172.21.15.191:0/634754345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- 172.21.15.191:0/634754345 >> 172.21.15.191:0/634754345 conn(0x55699aeb4400 msgr2=0x55699afa1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- 172.21.15.191:0/634754345 shutdown_connections 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- 172.21.15.191:0/634754345 wait complete. 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f5921a93a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55699af563c0 con 0x55699aeb4800 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.597+0000 7f591fa3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591fa3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38578/0 (socket says 172.21.15.191:38578) 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591fa3d700 1 -- 172.21.15.191:0/1840847976 learned_addr learned my addr 172.21.15.191:0/1840847976 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591fa3d700 1 -- 172.21.15.191:0/1840847976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aedda00 con 0x55699aeb4800 2024-04-27T04:34:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591fa3d700 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55699bc85e30 tx=0x55699afa5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591ca37700 1 -- 172.21.15.191:0/1840847976 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d8c0 con 0x55699aeb4800 2024-04-27T04:34:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591ca37700 1 -- 172.21.15.191:0/1840847976 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699af55c80 con 0x55699aeb4800 2024-04-27T04:34:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f591ca37700 1 -- 172.21.15.191:0/1840847976 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d4a0 con 0x55699aeb4800 2024-04-27T04:34:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55699aedd860 con 0x55699aeb4800 2024-04-27T04:34:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.598+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55699aedd6c0 con 0x55699aeb4800 2024-04-27T04:34:43.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f591ca37700 1 -- 172.21.15.191:0/1840847976 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55699bc56b00 con 0x55699aeb4800 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f5921a93a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f5921a93a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55699aef0640 con 0x55699aeb5400 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f591ca37700 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240000 0x55699b19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f591ca37700 1 -- 172.21.15.191:0/1840847976 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55699af2f180 con 0x55699aeb4800 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f591fa3d700 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240000 0x55699b19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f592023e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f592023e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38594/0 (socket says 172.21.15.191:38594) 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f592023e700 1 -- 172.21.15.191:0/991067091 learned_addr learned my addr 172.21.15.191:0/991067091 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f591fa3d700 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240000 0x55699b19d080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55699bd3ef00 tx=0x55699b1fe240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f592023e700 1 -- 172.21.15.191:0/991067091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aedc000 con 0x55699aeb5400 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.599+0000 7f592023e700 1 --2- 172.21.15.191:0/991067091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55699bd3e360 tx=0x55699b1ff0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf2df5632804d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f591a232700 1 -- 172.21.15.191:0/991067091 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d8c0 con 0x55699aeb5400 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f591a232700 1 -- 172.21.15.191:0/991067091 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699b1de480 con 0x55699aeb5400 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f591a232700 1 -- 172.21.15.191:0/991067091 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d4a0 con 0x55699aeb5400 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- 172.21.15.191:0/991067091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 msgr2=0x55699b19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 --2- 172.21.15.191:0/991067091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55699bd3e360 tx=0x55699b1ff0e0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- 172.21.15.191:0/991067091 shutdown_connections 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 --2- 172.21.15.191:0/991067091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- 172.21.15.191:0/991067091 >> 172.21.15.191:0/991067091 conn(0x55699aeb4c00 msgr2=0x55699afa1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- 172.21.15.191:0/991067091 shutdown_connections 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- 172.21.15.191:0/991067091 wait complete. 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.600+0000 7f5921a93a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55699aef08c0 con 0x55699aeb5400 2024-04-27T04:34:43.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f592023e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f592023e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38598/0 (socket says 172.21.15.191:38598) 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 learned_addr learned my addr 172.21.15.191:0/3842020318 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aedcea0 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f592023e700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55699bd3e3f0 tx=0x55699b1fecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f5918a2f700 1 -- 172.21.15.191:0/3842020318 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6db80 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f5918a2f700 1 -- 172.21.15.191:0/3842020318 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699b1de180 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55699aedc820 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f5918a2f700 1 -- 172.21.15.191:0/3842020318 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d8c0 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.601+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55699aedc9c0 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f5918a2f700 1 -- 172.21.15.191:0/3842020318 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55699bc56dc0 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f5918a2f700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240c00 0x55699b19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f5918a2f700 1 -- 172.21.15.191:0/3842020318 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55699af2f340 con 0x55699aeb5400 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f591f23c700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240c00 0x55699b19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55699aeb5800 0x55699b19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699aeb5c00 con 0x55699aeb5800 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.602+0000 7f592023e700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55699aeb5800 0x55699b19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f591f23c700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240c00 0x55699b19d600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55699b197890 tx=0x55699afa5800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f592023e700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55699aeb5800 0x55699b19db80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 realm 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241000 con 0x55699aeb5800 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241400 con 0x55699aeb5800 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.603+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241800 con 0x55699aeb5800 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55699b241c00 0x55699b19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241000 con 0x55699b241c00 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f591fa3d700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55699b241c00 0x55699b19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f591fa3d700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55699b241c00 0x55699b19e100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 20 searching for the correct realm 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.604+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241400 con 0x55699b241c00 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591fa3d700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55699b241400 0x55699b19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241000 con 0x55699b241400 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591f23c700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55699b241400 0x55699b19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591f23c700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55699b241400 0x55699b19ec00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591f23c700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55699b241000 0x55699b19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.605+0000 7f592023e700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55699b241000 0x55699b19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55699b241000 0x55699b19f180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.606+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241400 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.607+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241400 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.608+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.609+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241400 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.610+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.611+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241400 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.612+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.612+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241c00 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.612+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.612+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.612+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.612+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241400 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699aeb5c00 con 0x55699aeb5800 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a400 con 0x55699aeb5800 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.613+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a800 con 0x55699b241c00 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 zone default found 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 4 Realm: () 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 zonegroup default 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30ac00 con 0x55699b241000 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b000 con 0x55699b241c00 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b400 con 0x55699b241c00 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 started sync module instance, tier type = 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.614+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b800 con 0x55699b241400 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.615+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.616+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93912062049024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a800 con 0x55699b241400 2024-04-27T04:34:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.616+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b000 con 0x55699aeb5800 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.617+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93912062049024] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.617+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.617+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93912062050176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b400 con 0x55699aeb5800 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.617+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a400 con 0x55699b241c00 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.618+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93912062050176] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.618+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.618+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93912062051328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30ac00 con 0x55699b241c00 2024-04-27T04:34:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.618+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699aeb5c00 con 0x55699b241400 2024-04-27T04:34:43.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.619+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.619+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93912062051328] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.619+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93912062052480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b241800 con 0x55699b241400 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.619+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699aeb5800 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.620+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.620+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93912062052480] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.620+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93912062053632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b800 con 0x55699aeb5800 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.620+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a800 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.621+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.621+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93912062053632] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.621+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93912062054784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30bc00 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.621+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b000 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.622+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93912062054784] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.623+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.623+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93912062055936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b400 con 0x55699b241000 2024-04-27T04:34:43.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.623+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a000 con 0x55699b241000 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93912062055936] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93912062057088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b800 con 0x55699b241000 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=0 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=1 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=2 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=3 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=4 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=5 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=6 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93912062057088] v31'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 20 add_watcher() i=7 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.624+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a800 con 0x55699aeb5800 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.625+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.625+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93912063631360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30bc00 con 0x55699aeb5800 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93912063631360] v31'62 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5906a0b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5921a93a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5921a93a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5921a93a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.626+0000 7f5921a93a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55699af563c0 con 0x55699b30b400 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f591f23c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f591f23c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38608/0 (socket says 172.21.15.191:38608) 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f591f23c700 1 -- 172.21.15.191:0/2137686943 learned_addr learned my addr 172.21.15.191:0/2137686943 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f591f23c700 1 -- 172.21.15.191:0/2137686943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aedcb60 con 0x55699b30b400 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f591f23c700 1 --2- 172.21.15.191:0/2137686943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55699b25b440 tx=0x55699afa59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d5cb4af6d6ec63 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f590620a700 1 -- 172.21.15.191:0/2137686943 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699af6d760 con 0x55699b30b400 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f590620a700 1 -- 172.21.15.191:0/2137686943 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699b324000 con 0x55699b30b400 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.627+0000 7f590620a700 1 -- 172.21.15.191:0/2137686943 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699b326000 con 0x55699b30b400 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- 172.21.15.191:0/2137686943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 msgr2=0x55699b19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 --2- 172.21.15.191:0/2137686943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55699b25b440 tx=0x55699afa59e0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- 172.21.15.191:0/2137686943 shutdown_connections 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 --2- 172.21.15.191:0/2137686943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- 172.21.15.191:0/2137686943 >> 172.21.15.191:0/2137686943 conn(0x55699b30b000 msgr2=0x55699afa2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- 172.21.15.191:0/2137686943 shutdown_connections 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- 172.21.15.191:0/2137686943 wait complete. 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 Processor -- start 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- start start 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f5921a93a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55699af56500 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f591f23c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f591f23c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38622/0 (socket says 172.21.15.191:38622) 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.628+0000 7f591f23c700 1 -- 172.21.15.191:0/60974320 learned_addr learned my addr 172.21.15.191:0/60974320 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f591f23c700 1 -- 172.21.15.191:0/60974320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55699aedcd00 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f591f23c700 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55699b31f260 tx=0x55699afa58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f5904a07700 1 -- 172.21.15.191:0/60974320 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699b326160 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f5904a07700 1 -- 172.21.15.191:0/60974320 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55699b324180 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55699aedc340 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f5904a07700 1 -- 172.21.15.191:0/60974320 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55699b3262c0 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.629+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55699aeddd40 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.630+0000 7f5904a07700 1 -- 172.21.15.191:0/60974320 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55699bc57b80 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.630+0000 7f5904a07700 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b3a6000 0x55699b19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.630+0000 7f5904a07700 1 -- 172.21.15.191:0/60974320 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55699af2f500 con 0x55699b30b400 2024-04-27T04:34:43.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.630+0000 7f592023e700 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b3a6000 0x55699b19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.630+0000 7f5921a93a80 5 note: GC not initialized 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.630+0000 7f592023e700 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b3a6000 0x55699b19f700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55699b2ab830 tx=0x55699b32d320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5902202700 20 reqs_thread_entry: start 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f59001fe700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f58ff9dc700 20 reqs_thread_entry: start 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30b800 con 0x55699aeb5800 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:43.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.631+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b30a800 con 0x55699b241400 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.632+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.632+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b3a6800 con 0x55699aeb5800 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v31'2 uv2 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f5921a93a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f5921a93a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f5921a93a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 93912063633664 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b3a6c00 con 0x55699aeb5800 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 93912062053632 notify 133143986177 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55699b2add40 con 0x55699aeb5800 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f591a232700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986177 cookie 93912062053632 notifier 4316 bl.length()=533 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f591a232700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f591a232700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f591a232700 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b41a000 con 0x55699aeb5800 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.633+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(65 notify.4 [notify cookie 93912063633664 out=8b] v0'0 uv33 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.634+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.634+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 93912063633664 notify 133143986177 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55699b36fba0 con 0x55699aeb5800 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 1, 2024-04-27T04:34:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:43.652 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:43.652 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:43.652 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.634+0000 7f5906a0b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.634+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93912063631360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b3a7000 con 0x55699aeb5800 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.635+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 93912063631360] v31'63 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.635+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93912062049024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b3a7400 con 0x55699b241400 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.636+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 93912062049024] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.636+0000 7f5921a93a80 20 remove_watcher() i=0 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.636+0000 7f5921a93a80 2 removed watcher, disabling cache 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.636+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93912062050176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b3a7800 con 0x55699aeb5800 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.638+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 93912062050176] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.638+0000 7f5921a93a80 20 remove_watcher() i=1 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.638+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93912062051328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b3a7c00 con 0x55699b241c00 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.639+0000 7f591fa3d700 1 -- 172.21.15.191:0/3842020318 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 93912062051328] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff200 con 0x55699b241c00 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.639+0000 7f5921a93a80 20 remove_watcher() i=2 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.639+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93912062052480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b41a000 con 0x55699b241400 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.639+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 93912062052480] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2ff440 con 0x55699b241400 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.640+0000 7f5921a93a80 20 remove_watcher() i=3 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.640+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93912062053632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b41a800 con 0x55699aeb5800 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.641+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 93912062053632] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699aeb5800 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.641+0000 7f5921a93a80 20 remove_watcher() i=4 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.641+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93912062054784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b41ac00 con 0x55699b241000 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.641+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 93912062054784] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.641+0000 7f5921a93a80 20 remove_watcher() i=5 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.641+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93912062055936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b41b000 con 0x55699b241000 2024-04-27T04:34:43.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.642+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 93912062055936] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.643+0000 7f5921a93a80 20 remove_watcher() i=6 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.643+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93912062057088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55699b41b400 con 0x55699b241000 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f592023e700 1 -- 172.21.15.191:0/3842020318 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 93912062057088] v31'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55699b2feb40 con 0x55699b241000 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 20 remove_watcher() i=7 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55699b241000 msgr2=0x55699b19f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55699b241000 0x55699b19f180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55699aeb5800 msgr2=0x55699b19db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55699aeb5800 0x55699b19db80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55699b241400 msgr2=0x55699b19ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55699b241400 0x55699b19ec00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55699b241c00 msgr2=0x55699b19e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55699b241c00 0x55699b19e100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240c00 msgr2=0x55699b19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240c00 0x55699b19d600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55699b197890 tx=0x55699afa5800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 msgr2=0x55699b19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/3842020318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb5400 0x55699b19cb00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55699bd3e3f0 tx=0x55699b1fecc0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 reap_dead start 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f591f23c700 1 -- 172.21.15.191:0/3842020318 reap_dead start 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 shutdown_connections 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 >> 172.21.15.191:0/3842020318 conn(0x55699aeb4c00 msgr2=0x55699afa1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 shutdown_connections 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/3842020318 wait complete. 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b3a6000 msgr2=0x55699b19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b3a6000 0x55699b19f700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55699b2ab830 tx=0x55699b32d320 comp rx=0 tx=0).stop 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 msgr2=0x55699b19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55699b31f260 tx=0x55699afa58c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 shutdown_connections 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b3a6000 0x55699b19f700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 --2- 172.21.15.191:0/60974320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699b30b400 0x55699b19e680 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 >> 172.21.15.191:0/60974320 conn(0x55699b30b000 msgr2=0x55699afa2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 shutdown_connections 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.644+0000 7f5921a93a80 1 -- 172.21.15.191:0/60974320 wait complete. 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240000 msgr2=0x55699b19d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240000 0x55699b19d080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55699bd3ef00 tx=0x55699b1fe240 comp rx=0 tx=0).stop 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 msgr2=0x55699b19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55699bc85e30 tx=0x55699afa5920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 shutdown_connections 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55699b240000 0x55699b19d080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 --2- 172.21.15.191:0/1840847976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55699aeb4800 0x55699b19c580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 >> 172.21.15.191:0/1840847976 conn(0x55699aeb4400 msgr2=0x55699afa1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 shutdown_connections 2024-04-27T04:34:43.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.645+0000 7f5921a93a80 1 -- 172.21.15.191:0/1840847976 wait complete. 2024-04-27T04:34:43.662 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:43.662 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:43.662 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:43.662 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:43.708 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.708 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f06625faa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f06625faa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f751709500 con 0x55f7513dec00 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f06625faa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f751480a00 con 0x55f7513de800 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f06605a4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f065fda3700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55f7513dec00 0x55f7514ca800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55294/0 (socket says 172.21.15.191:55294) 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.707+0000 7f065fda3700 1 -- 172.21.15.191:0/4163923141 learned_addr learned my addr 172.21.15.191:0/4163923141 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06605a4700 1 -- 172.21.15.191:0/4163923141 >> v1:172.21.15.138:6789/0 conn(0x55f7513dec00 legacy=0x55f7514ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06605a4700 1 -- 172.21.15.191:0/4163923141 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751407d40 con 0x55f7513de800 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f065f5a2700 1 -- 172.21.15.191:0/4163923141 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963826052 0 0) 0x55f7521ac000 con 0x55f7513dec00 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06605a4700 1 --2- 172.21.15.191:0/4163923141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55f7516c19e0 tx=0x55f7514cf920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d976d10ca9702fc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f065f5a2700 1 -- 172.21.15.191:0/4163923141 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f7514974a0 con 0x55f7513de800 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f065f5a2700 1 -- 172.21.15.191:0/4163923141 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f751709800 con 0x55f7513de800 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f065f5a2700 1 -- 172.21.15.191:0/4163923141 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f751497600 con 0x55f7513de800 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 -- 172.21.15.191:0/4163923141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 msgr2=0x55f7516c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 --2- 172.21.15.191:0/4163923141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55f7516c19e0 tx=0x55f7514cf920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 -- 172.21.15.191:0/4163923141 shutdown_connections 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 --2- 172.21.15.191:0/4163923141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 -- 172.21.15.191:0/4163923141 >> 172.21.15.191:0/4163923141 conn(0x55f7513de400 msgr2=0x55f7514ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 -- 172.21.15.191:0/4163923141 shutdown_connections 2024-04-27T04:34:43.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.708+0000 7f06625faa80 1 -- 172.21.15.191:0/4163923141 wait complete. 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06625faa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06625faa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f751480500 con 0x55f7513de800 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06605a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06605a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38644/0 (socket says 172.21.15.191:38644) 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.709+0000 7f06605a4700 1 -- 172.21.15.191:0/3137419098 learned_addr learned my addr 172.21.15.191:0/3137419098 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06605a4700 1 -- 172.21.15.191:0/3137419098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751407ba0 con 0x55f7513de800 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06605a4700 1 --2- 172.21.15.191:0/3137419098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55f7521af800 tx=0x55f7514cf800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d24abd8b08b1dde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f065eda1700 1 -- 172.21.15.191:0/3137419098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f751497760 con 0x55f7513de800 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f065eda1700 1 -- 172.21.15.191:0/3137419098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f751709b00 con 0x55f7513de800 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f065eda1700 1 -- 172.21.15.191:0/3137419098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f7514971e0 con 0x55f7513de800 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 -- 172.21.15.191:0/3137419098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 msgr2=0x55f7516c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 --2- 172.21.15.191:0/3137419098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55f7521af800 tx=0x55f7514cf800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 -- 172.21.15.191:0/3137419098 shutdown_connections 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 --2- 172.21.15.191:0/3137419098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 -- 172.21.15.191:0/3137419098 >> 172.21.15.191:0/3137419098 conn(0x55f7513de400 msgr2=0x55f7514cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 -- 172.21.15.191:0/3137419098 shutdown_connections 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.710+0000 7f06625faa80 1 -- 172.21.15.191:0/3137419098 wait complete. 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06625faa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06625faa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7514803c0 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06605a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06605a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38650/0 (socket says 172.21.15.191:38650) 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06605a4700 1 -- 172.21.15.191:0/821883309 learned_addr learned my addr 172.21.15.191:0/821883309 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06605a4700 1 -- 172.21.15.191:0/821883309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751407a00 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06605a4700 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55f7521afec0 tx=0x55f7514cf920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f065d59e700 1 -- 172.21.15.191:0/821883309 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f7514974a0 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f065d59e700 1 -- 172.21.15.191:0/821883309 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f751709c80 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.711+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f751407860 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f065d59e700 1 -- 172.21.15.191:0/821883309 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f751497600 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7514076c0 con 0x55f7513de800 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f06625faa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f06625faa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f751480000 con 0x55f7513df400 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f065d59e700 1 -- 172.21.15.191:0/821883309 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f752180b00 con 0x55f7513de800 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f065d59e700 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176c000 0x55f7516c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f0660da5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f0660da5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38662/0 (socket says 172.21.15.191:38662) 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f0660da5700 1 -- 172.21.15.191:0/2030591311 learned_addr learned my addr 172.21.15.191:0/2030591311 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.712+0000 7f065d59e700 1 -- 172.21.15.191:0/821883309 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f751459180 con 0x55f7513de800 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06605a4700 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176c000 0x55f7516c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f0660da5700 1 -- 172.21.15.191:0/2030591311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751406000 con 0x55f7513df400 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f0660da5700 1 --2- 172.21.15.191:0/2030591311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55f7521af9b0 tx=0x55f751734b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997e14a6e305d643 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06605a4700 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176c000 0x55f7516c7080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55f752256f90 tx=0x55f751734c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f065ad99700 1 -- 172.21.15.191:0/2030591311 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f7514974a0 con 0x55f7513df400 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f065ad99700 1 -- 172.21.15.191:0/2030591311 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f75147fc80 con 0x55f7513df400 2024-04-27T04:34:43.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f065ad99700 1 -- 172.21.15.191:0/2030591311 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f751497600 con 0x55f7513df400 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 -- 172.21.15.191:0/2030591311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 msgr2=0x55f7516c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 --2- 172.21.15.191:0/2030591311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55f7521af9b0 tx=0x55f751734b40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 -- 172.21.15.191:0/2030591311 shutdown_connections 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 --2- 172.21.15.191:0/2030591311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 -- 172.21.15.191:0/2030591311 >> 172.21.15.191:0/2030591311 conn(0x55f7513dec00 msgr2=0x55f7514cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 -- 172.21.15.191:0/2030591311 shutdown_connections 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.713+0000 7f06625faa80 1 -- 172.21.15.191:0/2030591311 wait complete. 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f06625faa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f06625faa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75141a640 con 0x55f7513df400 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f0660da5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f0660da5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38678/0 (socket says 172.21.15.191:38678) 2024-04-27T04:34:43.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 learned_addr learned my addr 172.21.15.191:0/3232720238 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751406ea0 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f0660da5700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55f7522566c0 tx=0x55f7517345a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.714+0000 7f0659596700 1 -- 172.21.15.191:0/3232720238 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f751497760 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f0659596700 1 -- 172.21.15.191:0/3232720238 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f75147fb00 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f751406820 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f0659596700 1 -- 172.21.15.191:0/3232720238 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f7514974a0 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7514069c0 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f0659596700 1 -- 172.21.15.191:0/3232720238 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f752180dc0 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f0659596700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176cc00 0x55f7516c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f0659596700 1 -- 172.21.15.191:0/3232720238 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f751459340 con 0x55f7513df400 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.715+0000 7f065fda3700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176cc00 0x55f7516c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f7513df800 0x55f7516c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7513dfc00 con 0x55f7513df800 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f0660da5700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f7513df800 0x55f7516c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f065fda3700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176cc00 0x55f7516c7600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55f7521ae030 tx=0x55f75180e0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f0660da5700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f7513df800 0x55f7516c7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 20 realm 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d000 con 0x55f7513df800 2024-04-27T04:34:43.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.716+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d400 con 0x55f7513df800 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d800 con 0x55f7513df800 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f75176dc00 0x55f7516c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d000 con 0x55f75176dc00 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06605a4700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f75176dc00 0x55f7516c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.717+0000 7f06605a4700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f75176dc00 0x55f7516c8100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06625faa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06625faa80 20 searching for the correct realm 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d400 con 0x55f75176dc00 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06605a4700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f75176d400 0x55f7516c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d000 con 0x55f75176d400 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f065fda3700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f75176d400 0x55f7516c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.718+0000 7f065fda3700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f75176d400 0x55f7516c8c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.719+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.719+0000 7f065fda3700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f75176d000 0x55f7516c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.719+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.719+0000 7f0660da5700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f75176d000 0x55f7516c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.719+0000 7f0660da5700 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f75176d000 0x55f7516c9180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.720+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d400 2024-04-27T04:34:43.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.721+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d400 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.722+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.723+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.724+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.724+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.724+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.724+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.724+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.724+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d400 2024-04-27T04:34:43.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.725+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d400 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176dc00 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.726+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d400 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7513dfc00 con 0x55f7513df800 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830400 con 0x55f7513df800 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.727+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830800 con 0x55f75176dc00 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 zone default found 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 4 Realm: () 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 zonegroup default 2024-04-27T04:34:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830c00 con 0x55f75176d000 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831000 con 0x55f75176dc00 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.728+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831400 con 0x55f75176dc00 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.729+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.729+0000 7f06625faa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.729+0000 7f06625faa80 20 started sync module instance, tier type = 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.729+0000 7f06625faa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.729+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831800 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.730+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.730+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94520711305984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830800 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.730+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831000 con 0x55f7513df800 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.731+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94520711305984] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.731+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.731+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94520711307136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831400 con 0x55f7513df800 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.731+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830400 con 0x55f75176dc00 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.732+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.732+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94520711308288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830c00 con 0x55f75176dc00 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.732+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7513dfc00 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.732+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94520711307136] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.733+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94520711308288] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.733+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.733+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94520711309440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f75176d800 con 0x55f75176d400 2024-04-27T04:34:43.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.733+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f7513df800 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.734+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94520711309440] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.734+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.734+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94520711310592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831800 con 0x55f7513df800 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.734+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830800 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.736+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94520711310592] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.737+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.737+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94520711311744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831c00 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.737+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831000 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.739+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.739+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94520711312896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831400 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.739+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830000 con 0x55f75176d000 2024-04-27T04:34:43.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.739+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94520711311744] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94520711312896] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94520711314048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831800 con 0x55f75176d000 2024-04-27T04:34:43.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=0 2024-04-27T04:34:43.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=1 2024-04-27T04:34:43.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=2 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=3 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=4 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=5 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.742+0000 7f06625faa80 20 add_watcher() i=6 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.743+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94520711314048] v31'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.743+0000 7f06625faa80 20 add_watcher() i=7 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.743+0000 7f06625faa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.743+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830800 con 0x55f7513df800 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.744+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.744+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94520712871936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831c00 con 0x55f7513df800 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94520712871936] v31'64 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f0647572700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f06625faa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f06625faa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f06625faa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.745+0000 7f06625faa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75141a780 con 0x55f751831400 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f065fda3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f065fda3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38682/0 (socket says 172.21.15.191:38682) 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f065fda3700 1 -- 172.21.15.191:0/3626532858 learned_addr learned my addr 172.21.15.191:0/3626532858 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f065fda3700 1 -- 172.21.15.191:0/3626532858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751406b60 con 0x55f751831400 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f065fda3700 1 --2- 172.21.15.191:0/3626532858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55f7517ba4e0 tx=0x55f751735f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed9e7f3acb1364 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f0646d71700 1 -- 172.21.15.191:0/3626532858 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f75184e000 con 0x55f751831400 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f0646d71700 1 -- 172.21.15.191:0/3626532858 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f751854000 con 0x55f751831400 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.746+0000 7f0646d71700 1 -- 172.21.15.191:0/3626532858 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f75184e160 con 0x55f751831400 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- 172.21.15.191:0/3626532858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 msgr2=0x55f7516c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 --2- 172.21.15.191:0/3626532858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55f7517ba4e0 tx=0x55f751735f80 comp rx=0 tx=0).stop 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- 172.21.15.191:0/3626532858 shutdown_connections 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 --2- 172.21.15.191:0/3626532858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- 172.21.15.191:0/3626532858 >> 172.21.15.191:0/3626532858 conn(0x55f751831000 msgr2=0x55f7514cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- 172.21.15.191:0/3626532858 shutdown_connections 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- 172.21.15.191:0/3626532858 wait complete. 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 Processor -- start 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- start start 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f06625faa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f751480000 con 0x55f751831400 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f065fda3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f065fda3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38694/0 (socket says 172.21.15.191:38694) 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.747+0000 7f065fda3700 1 -- 172.21.15.191:0/3286250272 learned_addr learned my addr 172.21.15.191:0/3286250272 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f065fda3700 1 -- 172.21.15.191:0/3286250272 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f751406d00 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f065fda3700 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55f751851200 tx=0x55f75180e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f064556e700 1 -- 172.21.15.191:0/3286250272 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f75184e2c0 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f064556e700 1 -- 172.21.15.191:0/3286250272 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f751854180 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f751406340 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f064556e700 1 -- 172.21.15.191:0/3286250272 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f75184e000 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.748+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f751406b60 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.749+0000 7f064556e700 1 -- 172.21.15.191:0/3286250272 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f752181b80 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.749+0000 7f064556e700 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f7518d2000 0x55f7516c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.749+0000 7f064556e700 1 -- 172.21.15.191:0/3286250272 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f751459500 con 0x55f751831400 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.749+0000 7f0660da5700 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f7518d2000 0x55f7516c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.749+0000 7f06625faa80 5 note: GC not initialized 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.749+0000 7f0660da5700 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f7518d2000 0x55f7516c9700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f7517bb800 tx=0x55f75180e360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f0642d69700 20 reqs_thread_entry: start 2024-04-27T04:34:43.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f06625faa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f06625faa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f0640d65700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f0640543700 20 reqs_thread_entry: start 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f06625faa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.750+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751831800 con 0x55f7513df800 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.751+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.751+0000 7f06625faa80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:43.760 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 1, 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:43.761 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.762 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:43.763 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.751+0000 7f0647572700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.751+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94520712871936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f751830800 con 0x55f7513df800 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.752+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94520712871936] v31'65 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.752+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94520711305984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d2400 con 0x55f75176d400 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.755+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94520711305984] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.755+0000 7f06625faa80 20 remove_watcher() i=0 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.755+0000 7f06625faa80 2 removed watcher, disabling cache 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.755+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94520711307136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d2800 con 0x55f7513df800 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.757+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94520711307136] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.757+0000 7f06625faa80 20 remove_watcher() i=1 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.757+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94520711308288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d2c00 con 0x55f75176dc00 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.757+0000 7f06605a4700 1 -- 172.21.15.191:0/3232720238 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94520711308288] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817200 con 0x55f75176dc00 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.757+0000 7f06625faa80 20 remove_watcher() i=2 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.757+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94520711309440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d3000 con 0x55f75176d400 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.758+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94520711309440] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751817440 con 0x55f75176d400 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.758+0000 7f06625faa80 20 remove_watcher() i=3 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.758+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94520711310592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d3400 con 0x55f7513df800 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.760+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94520711310592] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f7513df800 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.760+0000 7f06625faa80 20 remove_watcher() i=4 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.760+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94520711311744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d3800 con 0x55f75176d000 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.761+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94520711311744] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.761+0000 7f06625faa80 20 remove_watcher() i=5 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.761+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94520711312896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d3c00 con 0x55f75176d000 2024-04-27T04:34:43.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.762+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94520711312896] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.762+0000 7f06625faa80 20 remove_watcher() i=6 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.762+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94520711314048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f7518d2800 con 0x55f75176d000 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f0660da5700 1 -- 172.21.15.191:0/3232720238 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94520711314048] v31'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f751816b40 con 0x55f75176d000 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 20 remove_watcher() i=7 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f75176d000 msgr2=0x55f7516c9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f75176d000 0x55f7516c9180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f7513df800 msgr2=0x55f7516c7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f7513df800 0x55f7516c7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f75176d400 msgr2=0x55f7516c8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f75176d400 0x55f7516c8c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f75176dc00 msgr2=0x55f7516c8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f75176dc00 0x55f7516c8100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176cc00 msgr2=0x55f7516c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176cc00 0x55f7516c7600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55f7521ae030 tx=0x55f75180e0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 msgr2=0x55f7516c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 --2- 172.21.15.191:0/3232720238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513df400 0x55f7516c6b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55f7522566c0 tx=0x55f7517345a0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 reap_dead start 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f065fda3700 1 -- 172.21.15.191:0/3232720238 reap_dead start 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 shutdown_connections 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 >> 172.21.15.191:0/3232720238 conn(0x55f7513dec00 msgr2=0x55f7514cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 shutdown_connections 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.763+0000 7f06625faa80 1 -- 172.21.15.191:0/3232720238 wait complete. 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f7518d2000 msgr2=0x55f7516c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f7518d2000 0x55f7516c9700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f7517bb800 tx=0x55f75180e360 comp rx=0 tx=0).stop 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 msgr2=0x55f7516c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55f751851200 tx=0x55f75180e4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 shutdown_connections 2024-04-27T04:34:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f7518d2000 0x55f7516c9700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/3286250272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f751831400 0x55f7516c8680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 >> 172.21.15.191:0/3286250272 conn(0x55f751831000 msgr2=0x55f7514cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 shutdown_connections 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/3286250272 wait complete. 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176c000 msgr2=0x55f7516c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176c000 0x55f7516c7080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55f752256f90 tx=0x55f751734c60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 msgr2=0x55f7516c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55f7521afec0 tx=0x55f7514cf920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 shutdown_connections 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f75176c000 0x55f7516c7080 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 --2- 172.21.15.191:0/821883309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7513de800 0x55f7516c6580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 >> 172.21.15.191:0/821883309 conn(0x55f7513de400 msgr2=0x55f7514cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 shutdown_connections 2024-04-27T04:34:43.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.764+0000 7f06625faa80 1 -- 172.21.15.191:0/821883309 wait complete. 2024-04-27T04:34:43.777 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:43.777 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-27T04:34:43.777 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-27T04:34:43.777 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-27T04:34:43.823 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.823 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6fb211fa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6fb211fa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f572195500 con 0x55f571e6ac00 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6fb211fa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f571f0ca00 con 0x55f571e6a800 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6fb00c9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6faf8c8700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55f571e6ac00 0x55f571f56800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55302/0 (socket says 172.21.15.191:55302) 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.822+0000 7f6faf8c8700 1 -- 172.21.15.191:0/1519958728 learned_addr learned my addr 172.21.15.191:0/1519958728 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb00c9700 1 -- 172.21.15.191:0/1519958728 >> v1:172.21.15.138:6789/0 conn(0x55f571e6ac00 legacy=0x55f571f56800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb00c9700 1 -- 172.21.15.191:0/1519958728 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e93d40 con 0x55f571e6a800 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6faf0c7700 1 -- 172.21.15.191:0/1519958728 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288602703 0 0) 0x55f572c38000 con 0x55f571e6ac00 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/1519958728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55f57214d9e0 tx=0x55f571f5b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc79efe68105ae5c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6faf0c7700 1 -- 172.21.15.191:0/1519958728 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f234a0 con 0x55f571e6a800 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6faf0c7700 1 -- 172.21.15.191:0/1519958728 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f572195800 con 0x55f571e6a800 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6faf0c7700 1 -- 172.21.15.191:0/1519958728 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f23600 con 0x55f571e6a800 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 -- 172.21.15.191:0/1519958728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 msgr2=0x55f572152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/1519958728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55f57214d9e0 tx=0x55f571f5b920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 -- 172.21.15.191:0/1519958728 shutdown_connections 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/1519958728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 -- 172.21.15.191:0/1519958728 >> 172.21.15.191:0/1519958728 conn(0x55f571e6a400 msgr2=0x55f571f56000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 -- 172.21.15.191:0/1519958728 shutdown_connections 2024-04-27T04:34:43.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.823+0000 7f6fb211fa80 1 -- 172.21.15.191:0/1519958728 wait complete. 2024-04-27T04:34:43.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb211fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb211fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571f0c500 con 0x55f571e6a800 2024-04-27T04:34:43.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38718/0 (socket says 172.21.15.191:38718) 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.824+0000 7f6fb00c9700 1 -- 172.21.15.191:0/2823832048 learned_addr learned my addr 172.21.15.191:0/2823832048 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 -- 172.21.15.191:0/2823832048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e93ba0 con 0x55f571e6a800 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/2823832048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55f572c3b800 tx=0x55f571f5b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dc9aa740ebe523 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fae8c6700 1 -- 172.21.15.191:0/2823832048 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f23760 con 0x55f571e6a800 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fae8c6700 1 -- 172.21.15.191:0/2823832048 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f572195b00 con 0x55f571e6a800 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fae8c6700 1 -- 172.21.15.191:0/2823832048 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f231e0 con 0x55f571e6a800 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2823832048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 msgr2=0x55f572152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/2823832048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55f572c3b800 tx=0x55f571f5b800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2823832048 shutdown_connections 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/2823832048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2823832048 >> 172.21.15.191:0/2823832048 conn(0x55f571e6a400 msgr2=0x55f571f57000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2823832048 shutdown_connections 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2823832048 wait complete. 2024-04-27T04:34:43.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571f0c3c0 con 0x55f571e6a800 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38726/0 (socket says 172.21.15.191:38726) 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 -- 172.21.15.191:0/353182213 learned_addr learned my addr 172.21.15.191:0/353182213 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 -- 172.21.15.191:0/353182213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e93a00 con 0x55f571e6a800 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f572c3bec0 tx=0x55f571f5b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fad0c3700 1 -- 172.21.15.191:0/353182213 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f234a0 con 0x55f571e6a800 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fad0c3700 1 -- 172.21.15.191:0/353182213 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f572195c80 con 0x55f571e6a800 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fad0c3700 1 -- 172.21.15.191:0/353182213 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f23600 con 0x55f571e6a800 2024-04-27T04:34:43.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.825+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f571e93860 con 0x55f571e6a800 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f571e936c0 con 0x55f571e6a800 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fad0c3700 1 -- 172.21.15.191:0/353182213 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f572c0cb00 con 0x55f571e6a800 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fad0c3700 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f571e6b400 0x55f572153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fad0c3700 1 -- 172.21.15.191:0/353182213 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f571ee5180 con 0x55f571e6a800 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb211fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb211fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571ea6640 con 0x55f571e6b800 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f571e6b400 0x55f572153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38742/0 (socket says 172.21.15.191:38742) 2024-04-27T04:34:43.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.826+0000 7f6fb00c9700 1 -- 172.21.15.191:0/2200205803 learned_addr learned my addr 172.21.15.191:0/2200205803 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb00c9700 1 -- 172.21.15.191:0/2200205803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e92000 con 0x55f571e6b800 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f571e6b400 0x55f572153080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55f572c3bf20 tx=0x55f5721b8b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/2200205803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55f572ce3050 tx=0x55f5721b8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8c78222e175f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6faa8be700 1 -- 172.21.15.191:0/2200205803 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f234a0 con 0x55f571e6b800 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6faa8be700 1 -- 172.21.15.191:0/2200205803 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f572195c80 con 0x55f571e6b800 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6faa8be700 1 -- 172.21.15.191:0/2200205803 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f23600 con 0x55f571e6b800 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2200205803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 msgr2=0x55f572152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/2200205803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55f572ce3050 tx=0x55f5721b8c60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2200205803 shutdown_connections 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/2200205803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2200205803 >> 172.21.15.191:0/2200205803 conn(0x55f571e6ac00 msgr2=0x55f571f57800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2200205803 shutdown_connections 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.827+0000 7f6fb211fa80 1 -- 172.21.15.191:0/2200205803 wait complete. 2024-04-27T04:34:43.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb211fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb211fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571ea68c0 con 0x55f571e6b800 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb00c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38756/0 (socket says 172.21.15.191:38756) 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 learned_addr learned my addr 172.21.15.191:0/295418312 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e92ea0 con 0x55f571e6b800 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55f572ce2930 tx=0x55f5721b8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fa90bb700 1 -- 172.21.15.191:0/295418312 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f23760 con 0x55f571e6b800 2024-04-27T04:34:43.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fa90bb700 1 -- 172.21.15.191:0/295418312 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f571f0be00 con 0x55f571e6b800 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f571e92820 con 0x55f571e6b800 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fa90bb700 1 -- 172.21.15.191:0/295418312 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f571f231e0 con 0x55f571e6b800 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.828+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f571e929c0 con 0x55f571e6b800 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6fa90bb700 1 -- 172.21.15.191:0/295418312 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f572c0cdc0 con 0x55f571e6b800 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6fa90bb700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572212c00 0x55f572153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6fa90bb700 1 -- 172.21.15.191:0/295418312 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f571ee5340 con 0x55f571e6b800 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572212c00 0x55f572153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f571e6bc00 0x55f572153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.829+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213000 con 0x55f571e6bc00 2024-04-27T04:34:43.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f571e6bc00 0x55f572153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572212c00 0x55f572153600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55f572c3a030 tx=0x55f572284000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f571e6bc00 0x55f572153b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 realm 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213400 con 0x55f571e6bc00 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213800 con 0x55f571e6bc00 2024-04-27T04:34:43.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.830+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213c00 con 0x55f571e6bc00 2024-04-27T04:34:43.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f572213000 0x55f572154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213400 con 0x55f572213000 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f572213000 0x55f572154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f572213000 0x55f572154100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 20 searching for the correct realm 2024-04-27T04:34:43.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.831+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298000 con 0x55f572213000 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6fb00c9700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f572298000 0x55f572154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213400 con 0x55f572298000 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f572298000 0x55f572154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f572298000 0x55f572154c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f572213400 0x55f572155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.832+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f572213400 0x55f572155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f572213400 0x55f572155180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.833+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572298000 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.834+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572298000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.835+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.836+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572298000 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.837+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.838+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572298000 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213400 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f571e6bc00 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572298000 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb211fa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb211fa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb211fa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb211fa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.839+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298400 con 0x55f571e6bc00 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298800 con 0x55f571e6bc00 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298c00 con 0x55f572213000 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 zone default found 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 4 Realm: () 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 zonegroup default 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299000 con 0x55f572213400 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.840+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299400 con 0x55f572213000 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298400 con 0x55f572213000 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 20 started sync module instance, tier type = 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.841+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298800 con 0x55f572298000 2024-04-27T04:34:43.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.842+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.842+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94512669301504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299000 con 0x55f572298000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.842+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299c00 con 0x55f571e6bc00 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.844+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.844+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94512669302656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213c00 con 0x55f571e6bc00 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.845+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572213800 con 0x55f572213000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.845+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94512669301504] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.845+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94512669302656] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.846+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.846+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94512669303808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299800 con 0x55f572213000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.846+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298800 con 0x55f572298000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.847+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94512669303808] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.847+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.847+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94512669304960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299000 con 0x55f572298000 2024-04-27T04:34:43.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.847+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298c00 con 0x55f571e6bc00 2024-04-27T04:34:43.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.850+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.850+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94512669306112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572299400 con 0x55f571e6bc00 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.850+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298400 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.850+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94512669304960] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.851+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94512669306112] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.851+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.851+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94512669307264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572212400 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.851+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2000 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.853+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94512669307264] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94512669308416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2400 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2800 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94512669308416] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94512669309568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2c00 con 0x55f572213400 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb211fa80 20 add_watcher() i=0 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb211fa80 20 add_watcher() i=1 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.857+0000 7f6fb211fa80 20 add_watcher() i=2 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 20 add_watcher() i=3 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 20 add_watcher() i=4 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 20 add_watcher() i=5 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 20 add_watcher() i=6 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94512669309568] v31'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 20 add_watcher() i=7 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.858+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a3000 con 0x55f571e6bc00 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.859+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.859+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94512670736384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a3400 con 0x55f571e6bc00 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.859+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94512670736384] v31'66 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6f97097700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6fb211fa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6fb211fa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6fb211fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6fb211fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571f0c3c0 con 0x55f5722a3c00 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6faf8c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6faf8c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38764/0 (socket says 172.21.15.191:38764) 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.860+0000 7f6faf8c8700 1 -- 172.21.15.191:0/140590456 learned_addr learned my addr 172.21.15.191:0/140590456 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6faf8c8700 1 -- 172.21.15.191:0/140590456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e92b60 con 0x55f5722a3c00 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/140590456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55f572231590 tx=0x55f5721b9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e71f1377b65155 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6f96896700 1 -- 172.21.15.191:0/140590456 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5722ba000 con 0x55f5722a3c00 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6f96896700 1 -- 172.21.15.191:0/140590456 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5722be000 con 0x55f5722a3c00 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6f96896700 1 -- 172.21.15.191:0/140590456 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5722ba160 con 0x55f5722a3c00 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6fb211fa80 1 -- 172.21.15.191:0/140590456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 msgr2=0x55f572154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/140590456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55f572231590 tx=0x55f5721b9f80 comp rx=0 tx=0).stop 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.861+0000 7f6fb211fa80 1 -- 172.21.15.191:0/140590456 shutdown_connections 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/140590456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 -- 172.21.15.191:0/140590456 >> 172.21.15.191:0/140590456 conn(0x55f5722a3800 msgr2=0x55f571f58000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 -- 172.21.15.191:0/140590456 shutdown_connections 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 -- 172.21.15.191:0/140590456 wait complete. 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 Processor -- start 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 -- start start 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6fb211fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571f0c500 con 0x55f5722a3c00 2024-04-27T04:34:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6faf8c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6faf8c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38776/0 (socket says 172.21.15.191:38776) 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6faf8c8700 1 -- 172.21.15.191:0/117536063 learned_addr learned my addr 172.21.15.191:0/117536063 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.862+0000 7f6faf8c8700 1 -- 172.21.15.191:0/117536063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f571e92d00 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6faf8c8700 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55f5722bcc00 tx=0x55f572284240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6f95093700 1 -- 172.21.15.191:0/117536063 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5722ba2c0 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6f95093700 1 -- 172.21.15.191:0/117536063 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5722be180 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f571e92340 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6f95093700 1 -- 172.21.15.191:0/117536063 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5722ba000 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f571e92b60 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.863+0000 7f6f95093700 1 -- 172.21.15.191:0/117536063 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f572c0db80 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.864+0000 7f6f95093700 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572299c00 0x55f572155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.864+0000 7f6f95093700 1 -- 172.21.15.191:0/117536063 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f571ee56c0 con 0x55f5722a3c00 2024-04-27T04:34:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.864+0000 7f6fb211fa80 5 note: GC not initialized 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.864+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572299c00 0x55f572155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.864+0000 7f6fb08ca700 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572299c00 0x55f572155700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f57220f7a0 tx=0x55f5722852c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6f9288e700 20 reqs_thread_entry: start 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6f9088a700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6f90068700 20 reqs_thread_entry: start 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572212400 con 0x55f571e6bc00 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:43.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.865+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298c00 con 0x55f572298000 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.866+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.866+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572298400 con 0x55f571e6bc00 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v31'3 uv3 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb211fa80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb211fa80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb211fa80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94512670738688 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f572212400 con 0x55f571e6bc00 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94512669306112 notify 133143986178 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55f5722b0000 con 0x55f571e6bc00 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6faa8be700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94512669306112 notifier 4332 bl.length()=533 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(65 notify.4 [notify cookie 94512670738688 out=8b] v0'0 uv39 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6faa8be700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6faa8be700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6faa8be700 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5723a8000 con 0x55f571e6bc00 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.867+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.868+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94512670738688 notify 133143986178 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f5722b01a0 con 0x55f571e6bc00 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:43.886 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:43.887 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:43.888 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:43.889 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:43.889 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:43.889 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:43.889 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.868+0000 7f6f97097700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.868+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94512670736384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2000 con 0x55f571e6bc00 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.869+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94512670736384] v31'67 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.869+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94512669301504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2400 con 0x55f572298000 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.870+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94512669301504] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.870+0000 7f6fb211fa80 20 remove_watcher() i=0 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.870+0000 7f6fb211fa80 2 removed watcher, disabling cache 2024-04-27T04:34:43.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.870+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94512669302656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2800 con 0x55f571e6bc00 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.871+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94512669302656] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.872+0000 7f6fb211fa80 20 remove_watcher() i=1 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.872+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94512669303808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a2c00 con 0x55f572213000 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.872+0000 7f6fb00c9700 1 -- 172.21.15.191:0/295418312 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94512669303808] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291200 con 0x55f572213000 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.873+0000 7f6fb211fa80 20 remove_watcher() i=2 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.873+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94512669304960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a3000 con 0x55f572298000 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.874+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94512669304960] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572291440 con 0x55f572298000 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.874+0000 7f6fb211fa80 20 remove_watcher() i=3 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.874+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94512669306112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5722a3400 con 0x55f571e6bc00 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.875+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94512669306112] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f571e6bc00 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.875+0000 7f6fb211fa80 20 remove_watcher() i=4 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.875+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94512669307264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5723a8800 con 0x55f572213400 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.875+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94512669307264] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.876+0000 7f6fb211fa80 20 remove_watcher() i=5 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.876+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94512669308416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5723a8c00 con 0x55f572213400 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.878+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94512669308416] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.878+0000 7f6fb211fa80 20 remove_watcher() i=6 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.878+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94512669309568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f5723a9000 con 0x55f572213400 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb08ca700 1 -- 172.21.15.191:0/295418312 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94512669309568] v31'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f572290b40 con 0x55f572213400 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 20 remove_watcher() i=7 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f572213400 msgr2=0x55f572155180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f572213400 0x55f572155180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f571e6bc00 msgr2=0x55f572153b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f571e6bc00 0x55f572153b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f572298000 msgr2=0x55f572154c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f572298000 0x55f572154c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f572213000 msgr2=0x55f572154100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f572213000 0x55f572154100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572212c00 msgr2=0x55f572153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572212c00 0x55f572153600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55f572c3a030 tx=0x55f572284000 comp rx=0 tx=0).stop 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 msgr2=0x55f572152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/295418312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6b800 0x55f572152b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55f572ce2930 tx=0x55f5721b8e40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 reap_dead start 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.879+0000 7f6faf8c8700 1 -- 172.21.15.191:0/295418312 reap_dead start 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 shutdown_connections 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 >> 172.21.15.191:0/295418312 conn(0x55f571e6ac00 msgr2=0x55f571f57800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 shutdown_connections 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/295418312 wait complete. 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572299c00 msgr2=0x55f572155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572299c00 0x55f572155700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f57220f7a0 tx=0x55f5722852c0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 msgr2=0x55f572154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55f5722bcc00 tx=0x55f572284240 comp rx=0 tx=0).stop 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 shutdown_connections 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f572299c00 0x55f572155700 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/117536063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5722a3c00 0x55f572154680 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 >> 172.21.15.191:0/117536063 conn(0x55f5722a3800 msgr2=0x55f571f58000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 shutdown_connections 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/117536063 wait complete. 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f571e6b400 msgr2=0x55f572153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f571e6b400 0x55f572153080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55f572c3bf20 tx=0x55f5721b8b40 comp rx=0 tx=0).stop 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 msgr2=0x55f572152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f572c3bec0 tx=0x55f571f5b920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 shutdown_connections 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f571e6b400 0x55f572153080 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 --2- 172.21.15.191:0/353182213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f571e6a800 0x55f572152580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 >> 172.21.15.191:0/353182213 conn(0x55f571e6a400 msgr2=0x55f571f57000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 shutdown_connections 2024-04-27T04:34:43.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.880+0000 7f6fb211fa80 1 -- 172.21.15.191:0/353182213 wait complete. 2024-04-27T04:34:43.899 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:43.899 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:43.899 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:43.899 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:43.944 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:43.945 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:43.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f879473ea80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f879473ea80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b8109f5500 con 0x55b8106cac00 2024-04-27T04:34:43.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f879473ea80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b81076ca00 con 0x55b8106ca800 2024-04-27T04:34:43.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f87926e8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f8791ee7700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55b8106cac00 0x55b8107b6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55314/0 (socket says 172.21.15.191:55314) 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.943+0000 7f8791ee7700 1 -- 172.21.15.191:0/2777217552 learned_addr learned my addr 172.21.15.191:0/2777217552 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87926e8700 1 -- 172.21.15.191:0/2777217552 >> v1:172.21.15.138:6789/0 conn(0x55b8106cac00 legacy=0x55b8107b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87926e8700 1 -- 172.21.15.191:0/2777217552 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f3d40 con 0x55b8106ca800 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87916e6700 1 -- 172.21.15.191:0/2777217552 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095078395 0 0) 0x55b811498000 con 0x55b8106cac00 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87926e8700 1 --2- 172.21.15.191:0/2777217552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55b8109ad9e0 tx=0x55b8107bb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eb1a22a6588de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87916e6700 1 -- 172.21.15.191:0/2777217552 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8107834a0 con 0x55b8106ca800 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87916e6700 1 -- 172.21.15.191:0/2777217552 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8109f5800 con 0x55b8106ca800 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f87916e6700 1 -- 172.21.15.191:0/2777217552 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810783600 con 0x55b8106ca800 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 -- 172.21.15.191:0/2777217552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 msgr2=0x55b8109b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 --2- 172.21.15.191:0/2777217552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55b8109ad9e0 tx=0x55b8107bb920 comp rx=0 tx=0).stop 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 -- 172.21.15.191:0/2777217552 shutdown_connections 2024-04-27T04:34:43.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 --2- 172.21.15.191:0/2777217552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 -- 172.21.15.191:0/2777217552 >> 172.21.15.191:0/2777217552 conn(0x55b8106ca400 msgr2=0x55b8107b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 -- 172.21.15.191:0/2777217552 shutdown_connections 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.944+0000 7f879473ea80 1 -- 172.21.15.191:0/2777217552 wait complete. 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.945+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.945+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.945+0000 7f879473ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.945+0000 7f879473ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81076c500 con 0x55b8106ca800 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38790/0 (socket says 172.21.15.191:38790) 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f87926e8700 1 -- 172.21.15.191:0/4022243114 learned_addr learned my addr 172.21.15.191:0/4022243114 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f87926e8700 1 -- 172.21.15.191:0/4022243114 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f3ba0 con 0x55b8106ca800 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f87926e8700 1 --2- 172.21.15.191:0/4022243114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55b81149b800 tx=0x55b8107bb800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58cefbc596e2d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f8790ee5700 1 -- 172.21.15.191:0/4022243114 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810783760 con 0x55b8106ca800 2024-04-27T04:34:43.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f8790ee5700 1 -- 172.21.15.191:0/4022243114 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8109f5b00 con 0x55b8106ca800 2024-04-27T04:34:43.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f8790ee5700 1 -- 172.21.15.191:0/4022243114 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8107831e0 con 0x55b8106ca800 2024-04-27T04:34:43.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 -- 172.21.15.191:0/4022243114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 msgr2=0x55b8109b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 --2- 172.21.15.191:0/4022243114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55b81149b800 tx=0x55b8107bb800 comp rx=0 tx=0).stop 2024-04-27T04:34:43.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 -- 172.21.15.191:0/4022243114 shutdown_connections 2024-04-27T04:34:43.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 --2- 172.21.15.191:0/4022243114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 -- 172.21.15.191:0/4022243114 >> 172.21.15.191:0/4022243114 conn(0x55b8106ca400 msgr2=0x55b8107b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 -- 172.21.15.191:0/4022243114 shutdown_connections 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.946+0000 7f879473ea80 1 -- 172.21.15.191:0/4022243114 wait complete. 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f879473ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f879473ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81076c3c0 con 0x55b8106ca800 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38800/0 (socket says 172.21.15.191:38800) 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f87926e8700 1 -- 172.21.15.191:0/71854582 learned_addr learned my addr 172.21.15.191:0/71854582 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.947+0000 7f87926e8700 1 -- 172.21.15.191:0/71854582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f3a00 con 0x55b8106ca800 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f87926e8700 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b81149bec0 tx=0x55b8107bb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f878f6e2700 1 -- 172.21.15.191:0/71854582 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8107834a0 con 0x55b8106ca800 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f878f6e2700 1 -- 172.21.15.191:0/71854582 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8109f5c80 con 0x55b8106ca800 2024-04-27T04:34:43.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b8106f3860 con 0x55b8106ca800 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f878f6e2700 1 -- 172.21.15.191:0/71854582 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810783600 con 0x55b8106ca800 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8106f36c0 con 0x55b8106ca800 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f878f6e2700 1 -- 172.21.15.191:0/71854582 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b81146cb00 con 0x55b8106ca800 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.948+0000 7f878f6e2700 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b8106cb400 0x55b8109b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f878f6e2700 1 -- 172.21.15.191:0/71854582 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b810745180 con 0x55b8106ca800 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f879473ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f879473ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81076c000 con 0x55b8106cb800 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f8792ee9700 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b8106cb400 0x55b8109b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38814/0 (socket says 172.21.15.191:38814) 2024-04-27T04:34:43.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f87926e8700 1 -- 172.21.15.191:0/3924943234 learned_addr learned my addr 172.21.15.191:0/3924943234 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f87926e8700 1 -- 172.21.15.191:0/3924943234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f2000 con 0x55b8106cb800 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f8792ee9700 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b8106cb400 0x55b8109b3080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55b81149bfb0 tx=0x55b810a1ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f87926e8700 1 --2- 172.21.15.191:0/3924943234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55b811542f90 tx=0x55b810a1ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa42f9ed7d6dd30 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f878cedd700 1 -- 172.21.15.191:0/3924943234 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8107834a0 con 0x55b8106cb800 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f878cedd700 1 -- 172.21.15.191:0/3924943234 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8109f5c80 con 0x55b8106cb800 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.949+0000 7f878cedd700 1 -- 172.21.15.191:0/3924943234 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810783600 con 0x55b8106cb800 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- 172.21.15.191:0/3924943234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 msgr2=0x55b8109b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 --2- 172.21.15.191:0/3924943234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55b811542f90 tx=0x55b810a1ac60 comp rx=0 tx=0).stop 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- 172.21.15.191:0/3924943234 shutdown_connections 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 --2- 172.21.15.191:0/3924943234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- 172.21.15.191:0/3924943234 >> 172.21.15.191:0/3924943234 conn(0x55b8106cac00 msgr2=0x55b8107b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- 172.21.15.191:0/3924943234 shutdown_connections 2024-04-27T04:34:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- 172.21.15.191:0/3924943234 wait complete. 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f879473ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b810706640 con 0x55b8106cb800 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f87926e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38826/0 (socket says 172.21.15.191:38826) 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.950+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 learned_addr learned my addr 172.21.15.191:0/1266383707 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f2ea0 con 0x55b8106cb800 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f87926e8700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b811542330 tx=0x55b810a1ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f878b6da700 1 -- 172.21.15.191:0/1266383707 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810783760 con 0x55b8106cb800 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f878b6da700 1 -- 172.21.15.191:0/1266383707 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b81076be00 con 0x55b8106cb800 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b8106f2820 con 0x55b8106cb800 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f878b6da700 1 -- 172.21.15.191:0/1266383707 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8107831e0 con 0x55b8106cb800 2024-04-27T04:34:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8106f29c0 con 0x55b8106cb800 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.951+0000 7f878b6da700 1 -- 172.21.15.191:0/1266383707 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b81146cdc0 con 0x55b8106cb800 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f878b6da700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810a70c00 0x55b8109b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f878b6da700 1 -- 172.21.15.191:0/1266383707 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b810745340 con 0x55b8106cb800 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f8791ee7700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810a70c00 0x55b8109b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b8106cbc00 0x55b8109b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71000 con 0x55b8106cbc00 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f8792ee9700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b8106cbc00 0x55b8109b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f8791ee7700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810a70c00 0x55b8109b3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55b81149a030 tx=0x55b810ae2000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.952+0000 7f8792ee9700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b8106cbc00 0x55b8109b3b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 realm 2024-04-27T04:34:43.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71400 con 0x55b8106cbc00 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71800 con 0x55b8106cbc00 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71c00 con 0x55b8106cbc00 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.953+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b810a71000 0x55b8109b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71400 con 0x55b810a71000 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f87926e8700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b810a71000 0x55b8109b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f87926e8700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b810a71000 0x55b8109b4100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 20 searching for the correct realm 2024-04-27T04:34:43.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa000 con 0x55b810a71000 2024-04-27T04:34:43.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f87926e8700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b810afa000 0x55b8109b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.954+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71400 con 0x55b810afa000 2024-04-27T04:34:43.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.955+0000 7f8791ee7700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b810afa000 0x55b8109b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.955+0000 7f8791ee7700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b810afa000 0x55b8109b4c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.955+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.955+0000 7f8791ee7700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b810a71400 0x55b8109b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.955+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.955+0000 7f8792ee9700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b810a71400 0x55b8109b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b810a71400 0x55b8109b5180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.956+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810afa000 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.957+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810afa000 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.958+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.959+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810afa000 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.960+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.961+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810afa000 2024-04-27T04:34:43.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71400 2024-04-27T04:34:43.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b8106cbc00 2024-04-27T04:34:43.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.962+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810afa000 2024-04-27T04:34:43.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa400 con 0x55b8106cbc00 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa800 con 0x55b8106cbc00 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:43.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afac00 con 0x55b810a71000 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 zone default found 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 4 Realm: () 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 zonegroup default 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.963+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb000 con 0x55b810a71400 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb400 con 0x55b810a71000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa400 con 0x55b810a71000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 started sync module instance, tier type = 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.964+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa800 con 0x55b810afa000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.965+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.965+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94249040911104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb000 con 0x55b810afa000 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.965+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afbc00 con 0x55b8106cbc00 2024-04-27T04:34:43.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.968+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.968+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94249040912256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71c00 con 0x55b8106cbc00 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.968+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71800 con 0x55b810a71000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.968+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94249040911104] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.969+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94249040912256] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.971+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.971+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94249040913408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb800 con 0x55b810a71000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.971+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa800 con 0x55b810afa000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.972+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.972+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94249040913408] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.972+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94249040914560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb000 con 0x55b810afa000 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.972+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afac00 con 0x55b8106cbc00 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.973+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.973+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94249040915712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb400 con 0x55b8106cbc00 2024-04-27T04:34:43.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.973+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa400 con 0x55b810a71400 2024-04-27T04:34:43.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.973+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94249040914560] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:43.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.974+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94249040915712] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.979+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.979+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94249040916864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a70400 con 0x55b810a71400 2024-04-27T04:34:43.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.979+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afbc00 con 0x55b810a71400 2024-04-27T04:34:43.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.979+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94249040916864] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.982+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.983+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94249040918016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02000 con 0x55b810a71400 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.983+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02400 con 0x55b810a71400 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.983+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.983+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94249040919168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02800 con 0x55b810a71400 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=0 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=1 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=2 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=3 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=4 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=5 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94249040918016] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=6 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94249040919168] v31'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 20 add_watcher() i=7 2024-04-27T04:34:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.984+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02c00 con 0x55b8106cbc00 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.985+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.985+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94249042329600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b03000 con 0x55b8106cbc00 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94249042329600] v31'68 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f87796b6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f879473ea80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f879473ea80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f879473ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f879473ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b810706780 con 0x55b810b03800 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.986+0000 7f8791ee7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8791ee7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38842/0 (socket says 172.21.15.191:38842) 2024-04-27T04:34:43.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8791ee7700 1 -- 172.21.15.191:0/1654472961 learned_addr learned my addr 172.21.15.191:0/1654472961 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8791ee7700 1 -- 172.21.15.191:0/1654472961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f2b60 con 0x55b810b03800 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8791ee7700 1 --2- 172.21.15.191:0/1654472961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55b810a535c0 tx=0x55b810a1bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5be909f2d1a8649 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8778eb5700 1 -- 172.21.15.191:0/1654472961 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810b1c000 con 0x55b810b03800 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8778eb5700 1 -- 172.21.15.191:0/1654472961 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b810b1e000 con 0x55b810b03800 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.987+0000 7f8778eb5700 1 -- 172.21.15.191:0/1654472961 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810b1c160 con 0x55b810b03800 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- 172.21.15.191:0/1654472961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 msgr2=0x55b8109b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 --2- 172.21.15.191:0/1654472961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55b810a535c0 tx=0x55b810a1bf80 comp rx=0 tx=0).stop 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- 172.21.15.191:0/1654472961 shutdown_connections 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 --2- 172.21.15.191:0/1654472961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- 172.21.15.191:0/1654472961 >> 172.21.15.191:0/1654472961 conn(0x55b810b03400 msgr2=0x55b8107b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- 172.21.15.191:0/1654472961 shutdown_connections 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- 172.21.15.191:0/1654472961 wait complete. 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 Processor -- start 2024-04-27T04:34:43.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- start start 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f879473ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81076c000 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f8791ee7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f8791ee7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38854/0 (socket says 172.21.15.191:38854) 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.988+0000 7f8791ee7700 1 -- 172.21.15.191:0/2028646160 learned_addr learned my addr 172.21.15.191:0/2028646160 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f8791ee7700 1 -- 172.21.15.191:0/2028646160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8106f2d00 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f8791ee7700 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55b810b09200 tx=0x55b810ae2240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f87776b2700 1 -- 172.21.15.191:0/2028646160 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810b1c2c0 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f87776b2700 1 -- 172.21.15.191:0/2028646160 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b810b1e180 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b8106f2340 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f87776b2700 1 -- 172.21.15.191:0/2028646160 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b810b1c000 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.989+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8106f2b60 con 0x55b810b03800 2024-04-27T04:34:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.990+0000 7f87776b2700 1 -- 172.21.15.191:0/2028646160 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b81146db80 con 0x55b810b03800 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.990+0000 7f87776b2700 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810afac00 0x55b8109b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.990+0000 7f87776b2700 1 -- 172.21.15.191:0/2028646160 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b8107456c0 con 0x55b810b03800 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.990+0000 7f8792ee9700 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810afac00 0x55b8109b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.990+0000 7f879473ea80 5 note: GC not initialized 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.990+0000 7f8792ee9700 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810afac00 0x55b8109b5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55b810a53f80 tx=0x55b810ae32c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f8774ead700 20 reqs_thread_entry: start 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f879473ea80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f879473ea80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f8772ea9700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f8772687700 20 reqs_thread_entry: start 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f879473ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71c00 con 0x55b8106cbc00 2024-04-27T04:34:43.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:43.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.991+0000 7f879473ea80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:43.999 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.000 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.001 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.992+0000 7f87796b6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.992+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94249042329600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a70400 con 0x55b8106cbc00 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.993+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94249042329600] v31'69 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.993+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94249040911104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afb400 con 0x55b810afa000 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.994+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94249040911104] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.994+0000 7f879473ea80 20 remove_watcher() i=0 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.994+0000 7f879473ea80 2 removed watcher, disabling cache 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.994+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94249040912256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810afa400 con 0x55b8106cbc00 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.995+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94249040912256] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.995+0000 7f879473ea80 20 remove_watcher() i=1 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.995+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94249040913408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810a71c00 con 0x55b810a71000 2024-04-27T04:34:44.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.996+0000 7f87926e8700 1 -- 172.21.15.191:0/1266383707 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94249040913408] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1200 con 0x55b810a71000 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.996+0000 7f879473ea80 20 remove_watcher() i=2 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.996+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94249040914560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02000 con 0x55b810afa000 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.997+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94249040914560] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af1440 con 0x55b810afa000 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.997+0000 7f879473ea80 20 remove_watcher() i=3 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.997+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94249040915712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02400 con 0x55b8106cbc00 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.998+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94249040915712] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b8106cbc00 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.998+0000 7f879473ea80 20 remove_watcher() i=4 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.998+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94249040916864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02800 con 0x55b810a71400 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.999+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94249040916864] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.999+0000 7f879473ea80 20 remove_watcher() i=5 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:43.999+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94249040918016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b02c00 con 0x55b810a71400 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.000+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94249040918016] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.000+0000 7f879473ea80 20 remove_watcher() i=6 2024-04-27T04:34:44.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.000+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94249040919168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b810b03000 con 0x55b810a71400 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f8792ee9700 1 -- 172.21.15.191:0/1266383707 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94249040919168] v31'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b810af0b40 con 0x55b810a71400 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 20 remove_watcher() i=7 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b810a71400 msgr2=0x55b8109b5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b810a71400 0x55b8109b5180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b8106cbc00 msgr2=0x55b8109b3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b8106cbc00 0x55b8109b3b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b810afa000 msgr2=0x55b8109b4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b810afa000 0x55b8109b4c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b810a71000 msgr2=0x55b8109b4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b810a71000 0x55b8109b4100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810a70c00 msgr2=0x55b8109b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810a70c00 0x55b8109b3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55b81149a030 tx=0x55b810ae2000 comp rx=0 tx=0).stop 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 msgr2=0x55b8109b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 --2- 172.21.15.191:0/1266383707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106cb800 0x55b8109b2b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b811542330 tx=0x55b810a1ae40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 reap_dead start 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f8791ee7700 1 -- 172.21.15.191:0/1266383707 reap_dead start 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 shutdown_connections 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 >> 172.21.15.191:0/1266383707 conn(0x55b8106cac00 msgr2=0x55b8107b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 shutdown_connections 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.001+0000 7f879473ea80 1 -- 172.21.15.191:0/1266383707 wait complete. 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810afac00 msgr2=0x55b8109b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810afac00 0x55b8109b5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55b810a53f80 tx=0x55b810ae32c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 msgr2=0x55b8109b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55b810b09200 tx=0x55b810ae2240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 shutdown_connections 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b810afac00 0x55b8109b5700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/2028646160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b810b03800 0x55b8109b4680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 >> 172.21.15.191:0/2028646160 conn(0x55b810b03400 msgr2=0x55b8107b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 shutdown_connections 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/2028646160 wait complete. 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b8106cb400 msgr2=0x55b8109b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b8106cb400 0x55b8109b3080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55b81149bfb0 tx=0x55b810a1ab40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 msgr2=0x55b8109b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b81149bec0 tx=0x55b8107bb920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 shutdown_connections 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b8106cb400 0x55b8109b3080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 --2- 172.21.15.191:0/71854582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8106ca800 0x55b8109b2580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 >> 172.21.15.191:0/71854582 conn(0x55b8106ca400 msgr2=0x55b8107b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 shutdown_connections 2024-04-27T04:34:44.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.002+0000 7f879473ea80 1 -- 172.21.15.191:0/71854582 wait complete. 2024-04-27T04:34:44.013 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.013 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-27T04:34:44.013 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-27T04:34:44.013 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-04-27T04:34:44.062 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.062 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe39825aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x555df5538c00 0x555df5820580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe39825aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555df5863500 con 0x555df5538800 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe39825aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x555df55daa00 con 0x555df5538c00 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe396204700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x555df5538800 0x555df5624800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55320/0 (socket says 172.21.15.191:55320) 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe395a03700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x555df5538c00 0x555df5820580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe396204700 1 -- 172.21.15.191:0/636544571 learned_addr learned my addr 172.21.15.191:0/636544571 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe395a03700 1 -- 172.21.15.191:0/636544571 >> v1:172.21.15.138:6789/0 conn(0x555df5538800 legacy=0x555df5624800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe395a03700 1 -- 172.21.15.191:0/636544571 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5561d40 con 0x555df5538c00 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.060+0000 7fe395202700 1 -- 172.21.15.191:0/636544571 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811676141 0 0) 0x555df6306000 con 0x555df5538800 2024-04-27T04:34:44.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe395a03700 1 --2- 172.21.15.191:0/636544571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538c00 0x555df5820580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x555df581b890 tx=0x555df5629920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c9bbe4c77a4d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe395202700 1 -- 172.21.15.191:0/636544571 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f14a0 con 0x555df5538c00 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe395202700 1 -- 172.21.15.191:0/636544571 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df5863800 con 0x555df5538c00 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe395202700 1 -- 172.21.15.191:0/636544571 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1600 con 0x555df5538c00 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 -- 172.21.15.191:0/636544571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538c00 msgr2=0x555df5820580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 --2- 172.21.15.191:0/636544571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538c00 0x555df5820580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x555df581b890 tx=0x555df5629920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 -- 172.21.15.191:0/636544571 shutdown_connections 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 --2- 172.21.15.191:0/636544571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538c00 0x555df5820580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 -- 172.21.15.191:0/636544571 >> 172.21.15.191:0/636544571 conn(0x555df5538400 msgr2=0x555df5624000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 -- 172.21.15.191:0/636544571 shutdown_connections 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.061+0000 7fe39825aa80 1 -- 172.21.15.191:0/636544571 wait complete. 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe39825aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe39825aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df55da500 con 0x555df5538800 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38872/0 (socket says 172.21.15.191:38872) 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.062+0000 7fe396204700 1 -- 172.21.15.191:0/1898220053 learned_addr learned my addr 172.21.15.191:0/1898220053 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe396204700 1 -- 172.21.15.191:0/1898220053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5561ba0 con 0x555df5538800 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe396204700 1 --2- 172.21.15.191:0/1898220053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x555df6309d70 tx=0x555df5629b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d76efd2412d958 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe394a01700 1 -- 172.21.15.191:0/1898220053 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f14a0 con 0x555df5538800 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe394a01700 1 -- 172.21.15.191:0/1898220053 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df5863b00 con 0x555df5538800 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe394a01700 1 -- 172.21.15.191:0/1898220053 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1600 con 0x555df5538800 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 -- 172.21.15.191:0/1898220053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 msgr2=0x555df5820580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 --2- 172.21.15.191:0/1898220053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x555df6309d70 tx=0x555df5629b60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 -- 172.21.15.191:0/1898220053 shutdown_connections 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 --2- 172.21.15.191:0/1898220053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 -- 172.21.15.191:0/1898220053 >> 172.21.15.191:0/1898220053 conn(0x555df5538400 msgr2=0x555df5625000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 -- 172.21.15.191:0/1898220053 shutdown_connections 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.063+0000 7fe39825aa80 1 -- 172.21.15.191:0/1898220053 wait complete. 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe39825aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe39825aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df55da3c0 con 0x555df5538800 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38874/0 (socket says 172.21.15.191:38874) 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe396204700 1 -- 172.21.15.191:0/500207805 learned_addr learned my addr 172.21.15.191:0/500207805 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe396204700 1 -- 172.21.15.191:0/500207805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5561a00 con 0x555df5538800 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe396204700 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x555df6309e60 tx=0x555df5629a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe3931fe700 1 -- 172.21.15.191:0/500207805 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f18c0 con 0x555df5538800 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe3931fe700 1 -- 172.21.15.191:0/500207805 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df55d9c80 con 0x555df5538800 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555df5561860 con 0x555df5538800 2024-04-27T04:34:44.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe3931fe700 1 -- 172.21.15.191:0/500207805 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f14a0 con 0x555df5538800 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.064+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555df55616c0 con 0x555df5538800 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe3931fe700 1 -- 172.21.15.191:0/500207805 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555df62dab00 con 0x555df5538800 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe39825aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe3931fe700 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5539400 0x555df5821080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe39825aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df5574640 con 0x555df5539800 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe3931fe700 1 -- 172.21.15.191:0/500207805 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x555df55b3180 con 0x555df5538800 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe396a05700 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5539400 0x555df5821080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38882/0 (socket says 172.21.15.191:38882) 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.065+0000 7fe396204700 1 -- 172.21.15.191:0/1800935978 learned_addr learned my addr 172.21.15.191:0/1800935978 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe396204700 1 -- 172.21.15.191:0/1800935978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5560000 con 0x555df5539800 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe396a05700 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5539400 0x555df5821080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x555df6309e90 tx=0x555df5884de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe396204700 1 --2- 172.21.15.191:0/1800935978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x555df63c6330 tx=0x555df5884240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69366b8542c2195 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe3909f9700 1 -- 172.21.15.191:0/1800935978 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1600 con 0x555df5539800 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe3909f9700 1 -- 172.21.15.191:0/1800935978 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df5862300 con 0x555df5539800 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe3909f9700 1 -- 172.21.15.191:0/1800935978 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1a20 con 0x555df5539800 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 -- 172.21.15.191:0/1800935978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 msgr2=0x555df5820b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 --2- 172.21.15.191:0/1800935978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x555df63c6330 tx=0x555df5884240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 -- 172.21.15.191:0/1800935978 shutdown_connections 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 --2- 172.21.15.191:0/1800935978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 -- 172.21.15.191:0/1800935978 >> 172.21.15.191:0/1800935978 conn(0x555df5538c00 msgr2=0x555df5625800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 -- 172.21.15.191:0/1800935978 shutdown_connections 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.066+0000 7fe39825aa80 1 -- 172.21.15.191:0/1800935978 wait complete. 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe39825aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe39825aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df55748c0 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe396204700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38888/0 (socket says 172.21.15.191:38888) 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 learned_addr learned my addr 172.21.15.191:0/281944557 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5560ea0 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe396204700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x555df58e4c30 tx=0x555df5884fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe38f1f6700 1 -- 172.21.15.191:0/281944557 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1600 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe38f1f6700 1 -- 172.21.15.191:0/281944557 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df55d9e00 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555df5560820 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe38f1f6700 1 -- 172.21.15.191:0/281944557 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1a20 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.067+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555df55609c0 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe38f1f6700 1 -- 172.21.15.191:0/281944557 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555df62dadc0 con 0x555df5539800 2024-04-27T04:34:44.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe38f1f6700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df58e8c00 0x555df5821600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe38f1f6700 1 -- 172.21.15.191:0/281944557 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x555df55b3340 con 0x555df5539800 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe395a03700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df58e8c00 0x555df5821600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555df5539c00 0x555df5821b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.068+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9000 con 0x555df5539c00 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe396a05700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555df5539c00 0x555df5821b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe395a03700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df58e8c00 0x555df5821600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555df581b890 tx=0x555df5629800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe396a05700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555df5539c00 0x555df5821b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 20 realm 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9400 con 0x555df5539c00 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.069+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9800 con 0x555df5539c00 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9c00 con 0x555df5539c00 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555df58e9000 0x555df5822100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9400 con 0x555df58e9000 2024-04-27T04:34:44.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe396204700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555df58e9000 0x555df5822100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.070+0000 7fe396204700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555df58e9000 0x555df5822100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe39825aa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe39825aa80 20 searching for the correct realm 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966000 con 0x555df58e9000 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe396204700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555df5966000 0x555df5822c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9400 con 0x555df5966000 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe395a03700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555df5966000 0x555df5822c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.071+0000 7fe395a03700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555df5966000 0x555df5822c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.072+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.072+0000 7fe395a03700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555df58e9400 0x555df5823180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.072+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.072+0000 7fe396a05700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555df58e9400 0x555df5823180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.072+0000 7fe396a05700 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555df58e9400 0x555df5823180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.073+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5966000 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.074+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5966000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.075+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.076+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.076+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.076+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.076+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.076+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.076+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.077+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.077+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.077+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.077+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.077+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.077+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5966000 2024-04-27T04:34:44.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.078+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5966000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.079+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9400 2024-04-27T04:34:44.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5539c00 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df5966000 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966400 con 0x555df5539c00 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966800 con 0x555df5539c00 2024-04-27T04:34:44.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.080+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966c00 con 0x555df58e9000 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 zone default found 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 4 Realm: () 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 zonegroup default 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967000 con 0x555df58e9400 2024-04-27T04:34:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967400 con 0x555df58e9000 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966400 con 0x555df58e9000 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 started sync module instance, tier type = 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.081+0000 7fe39825aa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.082+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966800 con 0x555df5966000 2024-04-27T04:34:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.083+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.083+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93862039223040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967000 con 0x555df5966000 2024-04-27T04:34:44.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.083+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967c00 con 0x555df5539c00 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.084+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93862039223040] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.084+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.084+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93862039224192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9c00 con 0x555df5539c00 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.084+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e9800 con 0x555df58e9000 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.085+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93862039224192] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.085+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.085+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93862039225344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967800 con 0x555df58e9000 2024-04-27T04:34:44.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.085+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966800 con 0x555df5966000 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.086+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93862039225344] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.086+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.086+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93862039226496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967000 con 0x555df5966000 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.086+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966c00 con 0x555df5539c00 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.087+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93862039226496] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.087+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.087+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93862039227648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5967400 con 0x555df5539c00 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.087+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966400 con 0x555df58e9400 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.088+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93862039227648] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.088+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.088+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93862039228800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e8400 con 0x555df58e9400 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.088+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972000 con 0x555df58e9400 2024-04-27T04:34:44.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.089+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93862039228800] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.089+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.089+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93862039229952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972400 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.089+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972800 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93862039231104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972c00 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=0 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=1 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=2 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=3 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=4 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=5 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93862039229952] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.090+0000 7fe39825aa80 20 add_watcher() i=6 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.091+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93862039231104] v31'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.091+0000 7fe39825aa80 20 add_watcher() i=7 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.091+0000 7fe39825aa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.091+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5973000 con 0x555df5539c00 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.091+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.091+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93862040674304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5973400 con 0x555df5539c00 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.092+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93862040674304] v31'70 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.092+0000 7fe37d1d2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.092+0000 7fe39825aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.092+0000 7fe39825aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.092+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe39825aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe39825aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df55da3c0 con 0x555df5973c00 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe395a03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe395a03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38904/0 (socket says 172.21.15.191:38904) 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe395a03700 1 -- 172.21.15.191:0/877495117 learned_addr learned my addr 172.21.15.191:0/877495117 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.093+0000 7fe395a03700 1 -- 172.21.15.191:0/877495117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5560b60 con 0x555df5973c00 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe395a03700 1 --2- 172.21.15.191:0/877495117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x555df59076b0 tx=0x555df5629980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de13aef2443a1de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe37c9d1700 1 -- 172.21.15.191:0/877495117 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df55f1760 con 0x555df5973c00 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe37c9d1700 1 -- 172.21.15.191:0/877495117 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df5988000 con 0x555df5973c00 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe37c9d1700 1 -- 172.21.15.191:0/877495117 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df5978160 con 0x555df5973c00 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 -- 172.21.15.191:0/877495117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 msgr2=0x555df5822680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 --2- 172.21.15.191:0/877495117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x555df59076b0 tx=0x555df5629980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 -- 172.21.15.191:0/877495117 shutdown_connections 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 --2- 172.21.15.191:0/877495117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 -- 172.21.15.191:0/877495117 >> 172.21.15.191:0/877495117 conn(0x555df5973800 msgr2=0x555df5626000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 -- 172.21.15.191:0/877495117 shutdown_connections 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.094+0000 7fe39825aa80 1 -- 172.21.15.191:0/877495117 wait complete. 2024-04-27T04:34:44.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe39825aa80 1 Processor -- start 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe39825aa80 1 -- start start 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe39825aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe39825aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df55da500 con 0x555df5973c00 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe395a03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe395a03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38912/0 (socket says 172.21.15.191:38912) 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe395a03700 1 -- 172.21.15.191:0/154020632 learned_addr learned my addr 172.21.15.191:0/154020632 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.095+0000 7fe395a03700 1 -- 172.21.15.191:0/154020632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df5560d00 con 0x555df5973c00 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe395a03700 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x555df597f260 tx=0x555df56298c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe37b1ce700 1 -- 172.21.15.191:0/154020632 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df59782c0 con 0x555df5973c00 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe37b1ce700 1 -- 172.21.15.191:0/154020632 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df5988180 con 0x555df5973c00 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555df5560340 con 0x555df5973c00 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe37b1ce700 1 -- 172.21.15.191:0/154020632 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555df5978420 con 0x555df5973c00 2024-04-27T04:34:44.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555df5561d40 con 0x555df5973c00 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.096+0000 7fe37b1ce700 1 -- 172.21.15.191:0/154020632 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555df62dbb80 con 0x555df5973c00 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.097+0000 7fe37b1ce700 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5967c00 0x555df5823700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.097+0000 7fe37b1ce700 1 -- 172.21.15.191:0/154020632 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x555df55b36c0 con 0x555df5973c00 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.097+0000 7fe396a05700 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5967c00 0x555df5823700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.097+0000 7fe39825aa80 5 note: GC not initialized 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.097+0000 7fe396a05700 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5967c00 0x555df5823700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x555df5907d70 tx=0x555df59952c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe3789c9700 20 reqs_thread_entry: start 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe3769c5700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe3761a3700 20 reqs_thread_entry: start 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:44.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e8400 con 0x555df5539c00 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.098+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966c00 con 0x555df58e9400 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.099+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.099+0000 7fe39825aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.099+0000 7fe39825aa80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.099+0000 7fe39825aa80 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:44.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.099+0000 7fe39825aa80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.099+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~459 in=459b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5966400 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~459] v31'4 uv4 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe39825aa80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe39825aa80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe39825aa80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 93862040676608 in=647b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df58e8400 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 93862039227648 notify 133143986179 ret 0) v3 ==== 677+0+0 (crc 0 0 0) 0x555df5a72000 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe3909f9700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 93862039227648 notifier 4348 bl.length()=635 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe3909f9700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe3909f9700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(65 notify.4 [notify cookie 93862040676608 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe3909f9700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a78000 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.100+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 93862040676608 notify 133143986179 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555df5a721a0 con 0x555df5539c00 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.101+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972000 con 0x555df58e9400 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe39825aa80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe39825aa80 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe39825aa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 93862040676608 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972400 con 0x555df58e9000 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 93862039225344 notify 133143986176 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x555df5908b60 con 0x555df58e9000 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe3901f8700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986176 cookie 93862039225344 notifier 4348 bl.length()=177 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe3901f8700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe3901f8700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(68 notify.2 [notify cookie 93862040676608 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe3901f8700 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a78800 con 0x555df58e9000 2024-04-27T04:34:44.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.102+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 93862040676608 notify 133143986176 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555df5908d00 con 0x555df58e9000 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:44.123 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.124 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.125 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.126 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.103+0000 7fe37d1d2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.103+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93862040674304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5973000 con 0x555df5539c00 2024-04-27T04:34:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.104+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93862040674304] v31'71 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.104+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93862039223040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5973400 con 0x555df5966000 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.105+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93862039223040] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.105+0000 7fe39825aa80 20 remove_watcher() i=0 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.105+0000 7fe39825aa80 2 removed watcher, disabling cache 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.105+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93862039224192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a78c00 con 0x555df5539c00 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.106+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93862039224192] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.106+0000 7fe39825aa80 20 remove_watcher() i=1 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.106+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93862039225344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a79000 con 0x555df58e9000 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.107+0000 7fe396204700 1 -- 172.21.15.191:0/281944557 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93862039225344] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957200 con 0x555df58e9000 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.107+0000 7fe39825aa80 20 remove_watcher() i=2 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.107+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93862039226496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a79400 con 0x555df5966000 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.108+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93862039226496] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5957440 con 0x555df5966000 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.108+0000 7fe39825aa80 20 remove_watcher() i=3 2024-04-27T04:34:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.108+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93862039227648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a79800 con 0x555df5539c00 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.109+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93862039227648] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df5539c00 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.109+0000 7fe39825aa80 20 remove_watcher() i=4 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.109+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93862039228800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5a79c00 con 0x555df58e9400 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.110+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93862039228800] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.110+0000 7fe39825aa80 20 remove_watcher() i=5 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.110+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93862039229952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972c00 con 0x555df58e9400 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.111+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93862039229952] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.111+0000 7fe39825aa80 20 remove_watcher() i=6 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.111+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93862039231104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555df5972800 con 0x555df58e9400 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe396a05700 1 -- 172.21.15.191:0/281944557 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93862039231104] v31'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555df5956b40 con 0x555df58e9400 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 20 remove_watcher() i=7 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555df58e9400 msgr2=0x555df5823180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555df58e9400 0x555df5823180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555df5539c00 msgr2=0x555df5821b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555df5539c00 0x555df5821b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555df5966000 msgr2=0x555df5822c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555df5966000 0x555df5822c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555df58e9000 msgr2=0x555df5822100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555df58e9000 0x555df5822100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df58e8c00 msgr2=0x555df5821600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df58e8c00 0x555df5821600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555df581b890 tx=0x555df5629800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 msgr2=0x555df5820b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/281944557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5539800 0x555df5820b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x555df58e4c30 tx=0x555df5884fc0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 reap_dead start 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe395a03700 1 -- 172.21.15.191:0/281944557 reap_dead start 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 shutdown_connections 2024-04-27T04:34:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 >> 172.21.15.191:0/281944557 conn(0x555df5538c00 msgr2=0x555df5625800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 shutdown_connections 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/281944557 wait complete. 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5967c00 msgr2=0x555df5823700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5967c00 0x555df5823700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x555df5907d70 tx=0x555df59952c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 msgr2=0x555df5822680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x555df597f260 tx=0x555df56298c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 shutdown_connections 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5967c00 0x555df5823700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 --2- 172.21.15.191:0/154020632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5973c00 0x555df5822680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 >> 172.21.15.191:0/154020632 conn(0x555df5973800 msgr2=0x555df5626000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.112+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 shutdown_connections 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/154020632 wait complete. 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5539400 msgr2=0x555df5821080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5539400 0x555df5821080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x555df6309e90 tx=0x555df5884de0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 msgr2=0x555df5820580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x555df6309e60 tx=0x555df5629a40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 shutdown_connections 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555df5539400 0x555df5821080 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 --2- 172.21.15.191:0/500207805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555df5538800 0x555df5820580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 >> 172.21.15.191:0/500207805 conn(0x555df5538400 msgr2=0x555df5625000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 shutdown_connections 2024-04-27T04:34:44.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.113+0000 7fe39825aa80 1 -- 172.21.15.191:0/500207805 wait complete. 2024-04-27T04:34:44.136 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.136 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.136 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.137 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:44.184 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.184 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.180+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.180+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.180+0000 7f19e2ee6a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b45621cc00 0x55b456504580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.180+0000 7f19e2ee6a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b456547500 con 0x55b45621c800 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.180+0000 7f19e2ee6a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b4562bea00 con 0x55b45621cc00 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19e0e90700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55b45621c800 0x55b456308800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55326/0 (socket says 172.21.15.191:55326) 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19e068f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b45621cc00 0x55b456504580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19e0e90700 1 -- 172.21.15.191:0/2703847727 learned_addr learned my addr 172.21.15.191:0/2703847727 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19dfe8e700 1 -- 172.21.15.191:0/2703847727 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281874252 0 0) 0x55b456fea000 con 0x55b45621c800 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19dfe8e700 1 -- 172.21.15.191:0/2703847727 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55b456547680 con 0x55b45621c800 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19e068f700 1 -- 172.21.15.191:0/2703847727 >> v1:172.21.15.138:6789/0 conn(0x55b45621c800 legacy=0x55b456308800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19e068f700 1 -- 172.21.15.191:0/2703847727 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456245d40 con 0x55b45621cc00 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19e068f700 1 --2- 172.21.15.191:0/2703847727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621cc00 0x55b456504580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55b4564ff890 tx=0x55b45630d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c3b0c8ec0bb5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19dfe8e700 1 -- 172.21.15.191:0/2703847727 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d54a0 con 0x55b45621cc00 2024-04-27T04:34:44.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19dfe8e700 1 -- 172.21.15.191:0/2703847727 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b456547680 con 0x55b45621cc00 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.181+0000 7f19dfe8e700 1 -- 172.21.15.191:0/2703847727 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d5600 con 0x55b45621cc00 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2703847727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621cc00 msgr2=0x55b456504580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2703847727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621cc00 0x55b456504580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55b4564ff890 tx=0x55b45630d980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2703847727 shutdown_connections 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2703847727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621cc00 0x55b456504580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2703847727 >> 172.21.15.191:0/2703847727 conn(0x55b45621c400 msgr2=0x55b456308000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2703847727 shutdown_connections 2024-04-27T04:34:44.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2703847727 wait complete. 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.182+0000 7f19e2ee6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19e2ee6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4562be500 con 0x55b45621c800 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19e0e90700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19e0e90700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38918/0 (socket says 172.21.15.191:38918) 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19e0e90700 1 -- 172.21.15.191:0/1415894552 learned_addr learned my addr 172.21.15.191:0/1415894552 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19e0e90700 1 -- 172.21.15.191:0/1415894552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456245ba0 con 0x55b45621c800 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19e0e90700 1 --2- 172.21.15.191:0/1415894552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b456fed770 tx=0x55b45630db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16c0bcc334823dc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19df68d700 1 -- 172.21.15.191:0/1415894552 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d51e0 con 0x55b45621c800 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19df68d700 1 -- 172.21.15.191:0/1415894552 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b456547b00 con 0x55b45621c800 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.183+0000 7f19df68d700 1 -- 172.21.15.191:0/1415894552 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d54a0 con 0x55b45621c800 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1415894552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 msgr2=0x55b456504580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/1415894552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b456fed770 tx=0x55b45630db60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1415894552 shutdown_connections 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/1415894552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1415894552 >> 172.21.15.191:0/1415894552 conn(0x55b45621c400 msgr2=0x55b456309000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1415894552 shutdown_connections 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1415894552 wait complete. 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e2ee6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4562be3c0 con 0x55b45621c800 2024-04-27T04:34:44.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e0e90700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e0e90700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38924/0 (socket says 172.21.15.191:38924) 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e0e90700 1 -- 172.21.15.191:0/3440481424 learned_addr learned my addr 172.21.15.191:0/3440481424 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.184+0000 7f19e0e90700 1 -- 172.21.15.191:0/3440481424 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456245a00 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19e0e90700 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b456fede30 tx=0x55b45630d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19dde8a700 1 -- 172.21.15.191:0/3440481424 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d5600 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19dde8a700 1 -- 172.21.15.191:0/3440481424 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b4562bdc80 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19dde8a700 1 -- 172.21.15.191:0/3440481424 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d51e0 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b456245860 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4562456c0 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.185+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19dde8a700 1 -- 172.21.15.191:0/3440481424 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b456fbeb00 con 0x55b45621c800 2024-04-27T04:34:44.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e2ee6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e2ee6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4562be000 con 0x55b45621d400 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19dde8a700 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565ae000 0x55b456505080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e1691700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19dde8a700 1 -- 172.21.15.191:0/3440481424 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b456297180 con 0x55b45621c800 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e1691700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38930/0 (socket says 172.21.15.191:38930) 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e1691700 1 -- 172.21.15.191:0/1809693382 learned_addr learned my addr 172.21.15.191:0/1809693382 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e0e90700 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565ae000 0x55b456505080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e1691700 1 -- 172.21.15.191:0/1809693382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456244000 con 0x55b45621d400 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e1691700 1 --2- 172.21.15.191:0/1809693382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55b456fede90 tx=0x55b45656cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13128008faddf2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19e0e90700 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565ae000 0x55b456505080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55b4570b0f00 tx=0x55b45656c240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19db685700 1 -- 172.21.15.191:0/1809693382 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d5600 con 0x55b45621d400 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19db685700 1 -- 172.21.15.191:0/1809693382 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b456546480 con 0x55b45621d400 2024-04-27T04:34:44.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.186+0000 7f19db685700 1 -- 172.21.15.191:0/1809693382 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d51e0 con 0x55b45621d400 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1809693382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 msgr2=0x55b456504b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/1809693382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55b456fede90 tx=0x55b45656cde0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1809693382 shutdown_connections 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/1809693382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1809693382 >> 172.21.15.191:0/1809693382 conn(0x55b45621cc00 msgr2=0x55b456309800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1809693382 shutdown_connections 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/1809693382 wait complete. 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e2ee6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b456258640 con 0x55b45621d400 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e1691700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e1691700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38940/0 (socket says 172.21.15.191:38940) 2024-04-27T04:34:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.187+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 learned_addr learned my addr 172.21.15.191:0/3058609688 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456244ea0 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19e1691700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55b4570b0450 tx=0x55b45630df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19d9e82700 1 -- 172.21.15.191:0/3058609688 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d5a20 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19d9e82700 1 -- 172.21.15.191:0/3058609688 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b456546180 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b456244820 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19d9e82700 1 -- 172.21.15.191:0/3058609688 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d5600 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.188+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4562449c0 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19d9e82700 1 -- 172.21.15.191:0/3058609688 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b456fbedc0 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19d9e82700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565aec00 0x55b456505600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19d9e82700 1 -- 172.21.15.191:0/3058609688 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b456297340 con 0x55b45621d400 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e068f700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565aec00 0x55b456505600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b45621d800 0x55b456505b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45621dc00 con 0x55b45621d800 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e1691700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b45621d800 0x55b456505b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e068f700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565aec00 0x55b456505600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55b4564ff890 tx=0x55b45630d800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.189+0000 7f19e1691700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b45621d800 0x55b456505b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 realm 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af000 con 0x55b45621d800 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af400 con 0x55b45621d800 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af800 con 0x55b45621d800 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b4565afc00 0x55b456506100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af000 con 0x55b4565afc00 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.190+0000 7f19e0e90700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b4565afc00 0x55b456506100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e0e90700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b4565afc00 0x55b456506100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e2ee6a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e2ee6a80 20 searching for the correct realm 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af400 con 0x55b4565afc00 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e0e90700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b4565af400 0x55b456506c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af000 con 0x55b4565af400 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.191+0000 7f19e068f700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b4565af400 0x55b456506c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.192+0000 7f19e068f700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b4565af400 0x55b456506c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.192+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.192+0000 7f19e068f700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b4565af000 0x55b456507180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.192+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.192+0000 7f19e1691700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b4565af000 0x55b456507180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.192+0000 7f19e1691700 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b4565af000 0x55b456507180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.193+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af400 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.194+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af400 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.195+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.196+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af400 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.197+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af400 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565afc00 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.198+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af400 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45621dc00 con 0x55b45621d800 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.199+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e400 con 0x55b45621d800 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e800 con 0x55b4565afc00 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 zone default found 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 4 Realm: () 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 zonegroup default 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666ec00 con 0x55b4565af000 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f000 con 0x55b4565afc00 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.200+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f400 con 0x55b4565afc00 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.201+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.201+0000 7f19e2ee6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.201+0000 7f19e2ee6a80 20 started sync module instance, tier type = 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.201+0000 7f19e2ee6a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.201+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f800 con 0x55b4565af400 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.202+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.202+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94233030540032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e800 con 0x55b4565af400 2024-04-27T04:34:44.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.202+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f000 con 0x55b45621d800 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.203+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94233030540032] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.203+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.203+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94233030541184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f400 con 0x55b45621d800 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.203+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e400 con 0x55b4565afc00 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.204+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.204+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94233030542336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666ec00 con 0x55b4565afc00 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.204+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45621dc00 con 0x55b4565af400 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.204+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94233030541184] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.205+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94233030542336] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.205+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.205+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94233030543488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b4565af800 con 0x55b4565af400 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.205+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b45621d800 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.206+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94233030543488] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.206+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.206+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94233030544640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f800 con 0x55b45621d800 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.206+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e800 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.207+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94233030544640] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.207+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.207+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94233030545792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666fc00 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.207+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f000 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.208+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.208+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94233030546944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f400 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.208+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e000 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94233030545792] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94233030546944] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94233030548096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f800 con 0x55b4565af000 2024-04-27T04:34:44.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=0 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=1 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=2 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=3 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=4 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=5 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.210+0000 7f19e2ee6a80 20 add_watcher() i=6 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.211+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94233030548096] v31'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.211+0000 7f19e2ee6a80 20 add_watcher() i=7 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.211+0000 7f19e2ee6a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.211+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e800 con 0x55b45621d800 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.211+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.211+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94233032105984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666fc00 con 0x55b45621d800 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.212+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94233032105984] v31'72 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.212+0000 7f19c7e5e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.212+0000 7f19e2ee6a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.212+0000 7f19e2ee6a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.212+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e2ee6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e2ee6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b456258780 con 0x55b45666f400 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e068f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e068f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38954/0 (socket says 172.21.15.191:38954) 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e068f700 1 -- 172.21.15.191:0/2655273610 learned_addr learned my addr 172.21.15.191:0/2655273610 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e068f700 1 -- 172.21.15.191:0/2655273610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456244b60 con 0x55b45666f400 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19e068f700 1 --2- 172.21.15.191:0/2655273610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55b456611800 tx=0x55b45630d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b404005615601c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19c765d700 1 -- 172.21.15.191:0/2655273610 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b4562d5760 con 0x55b45666f400 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19c765d700 1 -- 172.21.15.191:0/2655273610 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b456684000 con 0x55b45666f400 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.213+0000 7f19c765d700 1 -- 172.21.15.191:0/2655273610 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b45668e000 con 0x55b45666f400 2024-04-27T04:34:44.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2655273610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 msgr2=0x55b456506680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2655273610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55b456611800 tx=0x55b45630d9e0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2655273610 shutdown_connections 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2655273610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2655273610 >> 172.21.15.191:0/2655273610 conn(0x55b45666f000 msgr2=0x55b45630a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2655273610 shutdown_connections 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2655273610 wait complete. 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 Processor -- start 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.214+0000 7f19e2ee6a80 1 -- start start 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e2ee6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e2ee6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4562be000 con 0x55b45666f400 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e068f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e068f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38960/0 (socket says 172.21.15.191:38960) 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e068f700 1 -- 172.21.15.191:0/2718583676 learned_addr learned my addr 172.21.15.191:0/2718583676 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e068f700 1 -- 172.21.15.191:0/2718583676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b456244d00 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e068f700 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55b456683260 tx=0x55b45630d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19c5e5a700 1 -- 172.21.15.191:0/2718583676 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b45668e160 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19c5e5a700 1 -- 172.21.15.191:0/2718583676 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b456684180 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b456244340 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19c5e5a700 1 -- 172.21.15.191:0/2718583676 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b45668e2c0 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.215+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b456245d40 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.216+0000 7f19c5e5a700 1 -- 172.21.15.191:0/2718583676 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b456fbfb80 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.216+0000 7f19c5e5a700 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b45670e000 0x55b456507700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.216+0000 7f19c5e5a700 1 -- 172.21.15.191:0/2718583676 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b456297500 con 0x55b45666f400 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.216+0000 7f19e2ee6a80 5 note: GC not initialized 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.216+0000 7f19e1691700 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b45670e000 0x55b456507700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.216+0000 7f19e1691700 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b45670e000 0x55b456507700 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55b456611470 tx=0x55b45668f320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19c3655700 20 reqs_thread_entry: start 2024-04-27T04:34:44.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19e2ee6a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19e2ee6a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19c1651700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19c0e2f700 20 reqs_thread_entry: start 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19e2ee6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.217+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666f800 con 0x55b45621d800 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.218+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.218+0000 7f19e2ee6a80 20 rados_obj.operate() r=0 bl.length=459 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.233 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.234 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.235 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.236 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.219+0000 7f19c7e5e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.219+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94233032105984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45666e800 con 0x55b45621d800 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.220+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94233032105984] v31'73 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.220+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94233030540032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670e400 con 0x55b4565af400 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.220+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94233030540032] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.221+0000 7f19e2ee6a80 20 remove_watcher() i=0 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.221+0000 7f19e2ee6a80 2 removed watcher, disabling cache 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.221+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94233030541184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670e800 con 0x55b45621d800 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.222+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94233030541184] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.222+0000 7f19e2ee6a80 20 remove_watcher() i=1 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.222+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94233030542336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670ec00 con 0x55b4565afc00 2024-04-27T04:34:44.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.222+0000 7f19e0e90700 1 -- 172.21.15.191:0/3058609688 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94233030542336] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667200 con 0x55b4565afc00 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.223+0000 7f19e2ee6a80 20 remove_watcher() i=2 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.223+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94233030543488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670f000 con 0x55b4565af400 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.223+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94233030543488] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456667440 con 0x55b4565af400 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.224+0000 7f19e2ee6a80 20 remove_watcher() i=3 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.224+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94233030544640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670f400 con 0x55b45621d800 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.224+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94233030544640] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b45621d800 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.224+0000 7f19e2ee6a80 20 remove_watcher() i=4 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.224+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94233030545792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670f800 con 0x55b4565af000 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.225+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94233030545792] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.225+0000 7f19e2ee6a80 20 remove_watcher() i=5 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.225+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94233030546944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670fc00 con 0x55b4565af000 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.226+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94233030546944] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.226+0000 7f19e2ee6a80 20 remove_watcher() i=6 2024-04-27T04:34:44.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.226+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94233030548096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b45670e800 con 0x55b4565af000 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e1691700 1 -- 172.21.15.191:0/3058609688 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94233030548096] v31'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b456666b40 con 0x55b4565af000 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 20 remove_watcher() i=7 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b4565af000 msgr2=0x55b456507180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b4565af000 0x55b456507180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b45621d800 msgr2=0x55b456505b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b45621d800 0x55b456505b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b4565af400 msgr2=0x55b456506c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b4565af400 0x55b456506c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b4565afc00 msgr2=0x55b456506100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b4565afc00 0x55b456506100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565aec00 msgr2=0x55b456505600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565aec00 0x55b456505600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55b4564ff890 tx=0x55b45630d800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 msgr2=0x55b456504b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3058609688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621d400 0x55b456504b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55b4570b0450 tx=0x55b45630df20 comp rx=0 tx=0).stop 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 reap_dead start 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e068f700 1 -- 172.21.15.191:0/3058609688 reap_dead start 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 shutdown_connections 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 >> 172.21.15.191:0/3058609688 conn(0x55b45621cc00 msgr2=0x55b456309800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.227+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 shutdown_connections 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3058609688 wait complete. 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b45670e000 msgr2=0x55b456507700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b45670e000 0x55b456507700 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55b456611470 tx=0x55b45668f320 comp rx=0 tx=0).stop 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 msgr2=0x55b456506680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55b456683260 tx=0x55b45630d8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 shutdown_connections 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b45670e000 0x55b456507700 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/2718583676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45666f400 0x55b456506680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 >> 172.21.15.191:0/2718583676 conn(0x55b45666f000 msgr2=0x55b45630a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 shutdown_connections 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/2718583676 wait complete. 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565ae000 msgr2=0x55b456505080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565ae000 0x55b456505080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55b4570b0f00 tx=0x55b45656c240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 msgr2=0x55b456504580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b456fede30 tx=0x55b45630d920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 shutdown_connections 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b4565ae000 0x55b456505080 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 --2- 172.21.15.191:0/3440481424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b45621c800 0x55b456504580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 >> 172.21.15.191:0/3440481424 conn(0x55b45621c400 msgr2=0x55b456309000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 shutdown_connections 2024-04-27T04:34:44.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.228+0000 7f19e2ee6a80 1 -- 172.21.15.191:0/3440481424 wait complete. 2024-04-27T04:34:44.246 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.247 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-27T04:34:44.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-27T04:34:44.247 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-04-27T04:34:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8715b28a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5627b3512c00 0x5627b37fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8715b28a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5627b383d500 con 0x5627b3512800 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8715b28a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5627b35b4a00 con 0x5627b3512c00 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f87132d1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5627b3512c00 0x5627b37fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8713ad2700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5627b3512800 0x5627b35fe800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55338/0 (socket says 172.21.15.191:55338) 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f8713ad2700 1 -- 172.21.15.191:0/4157949142 learned_addr learned my addr 172.21.15.191:0/4157949142 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f87132d1700 1 -- 172.21.15.191:0/4157949142 >> v1:172.21.15.138:6789/0 conn(0x5627b3512800 legacy=0x5627b35fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.292+0000 7f87132d1700 1 -- 172.21.15.191:0/4157949142 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353bd40 con 0x5627b3512c00 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f87132d1700 1 --2- 172.21.15.191:0/4157949142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512c00 0x5627b37fa580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5627b37f5890 tx=0x5627b3603920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d52a0578215d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8712ad0700 1 -- 172.21.15.191:0/4157949142 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb4a0 con 0x5627b3512c00 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8712ad0700 1 -- 172.21.15.191:0/4157949142 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b383d800 con 0x5627b3512c00 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8712ad0700 1 -- 172.21.15.191:0/4157949142 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb600 con 0x5627b3512c00 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 -- 172.21.15.191:0/4157949142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512c00 msgr2=0x5627b37fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4157949142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512c00 0x5627b37fa580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5627b37f5890 tx=0x5627b3603920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 -- 172.21.15.191:0/4157949142 shutdown_connections 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4157949142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512c00 0x5627b37fa580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 -- 172.21.15.191:0/4157949142 >> 172.21.15.191:0/4157949142 conn(0x5627b3512400 msgr2=0x5627b35fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 -- 172.21.15.191:0/4157949142 shutdown_connections 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.293+0000 7f8715b28a80 1 -- 172.21.15.191:0/4157949142 wait complete. 2024-04-27T04:34:44.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8715b28a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8715b28a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b35b4500 con 0x5627b3512800 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8713ad2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8713ad2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38978/0 (socket says 172.21.15.191:38978) 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8713ad2700 1 -- 172.21.15.191:0/3865069890 learned_addr learned my addr 172.21.15.191:0/3865069890 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.294+0000 7f8713ad2700 1 -- 172.21.15.191:0/3865069890 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353bba0 con 0x5627b3512800 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8713ad2700 1 --2- 172.21.15.191:0/3865069890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5627b42eb860 tx=0x5627b3603b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95584fa3dc4a5624 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f87122cf700 1 -- 172.21.15.191:0/3865069890 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb1e0 con 0x5627b3512800 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f87122cf700 1 -- 172.21.15.191:0/3865069890 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b383db00 con 0x5627b3512800 2024-04-27T04:34:44.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f87122cf700 1 -- 172.21.15.191:0/3865069890 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb4a0 con 0x5627b3512800 2024-04-27T04:34:44.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 -- 172.21.15.191:0/3865069890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 msgr2=0x5627b37fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 --2- 172.21.15.191:0/3865069890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5627b42eb860 tx=0x5627b3603b60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 -- 172.21.15.191:0/3865069890 shutdown_connections 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 --2- 172.21.15.191:0/3865069890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 -- 172.21.15.191:0/3865069890 >> 172.21.15.191:0/3865069890 conn(0x5627b3512400 msgr2=0x5627b35ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 -- 172.21.15.191:0/3865069890 shutdown_connections 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 -- 172.21.15.191:0/3865069890 wait complete. 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.295+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8715b28a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8715b28a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b35b43c0 con 0x5627b3512800 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8713ad2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8713ad2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:38984/0 (socket says 172.21.15.191:38984) 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8713ad2700 1 -- 172.21.15.191:0/3258485068 learned_addr learned my addr 172.21.15.191:0/3258485068 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8713ad2700 1 -- 172.21.15.191:0/3258485068 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353ba00 con 0x5627b3512800 2024-04-27T04:34:44.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8713ad2700 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5627b42eba40 tx=0x5627b3603a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8710acc700 1 -- 172.21.15.191:0/3258485068 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb600 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8710acc700 1 -- 172.21.15.191:0/3258485068 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b35b3c80 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8710acc700 1 -- 172.21.15.191:0/3258485068 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb1e0 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627b353b860 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.296+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627b353b6c0 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8710acc700 1 -- 172.21.15.191:0/3258485068 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5627b42b4b00 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8715b28a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8715b28a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b354e640 con 0x5627b3513400 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8710acc700 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4000 0x5627b37fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8710acc700 1 -- 172.21.15.191:0/3258485068 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5627b358d180 con 0x5627b3512800 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f87142d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f87142d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39000/0 (socket says 172.21.15.191:39000) 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f87142d3700 1 -- 172.21.15.191:0/2877123276 learned_addr learned my addr 172.21.15.191:0/2877123276 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8713ad2700 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4000 0x5627b37fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f87142d3700 1 -- 172.21.15.191:0/2877123276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353a000 con 0x5627b3513400 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f87142d3700 1 --2- 172.21.15.191:0/2877123276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5627b4390450 tx=0x5627b385ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18eec7896555b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.297+0000 7f8713ad2700 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4000 0x5627b37fb080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5627b4390ab0 tx=0x5627b385e240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f870e2c7700 1 -- 172.21.15.191:0/2877123276 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb600 con 0x5627b3513400 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f870e2c7700 1 -- 172.21.15.191:0/2877123276 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b383c300 con 0x5627b3513400 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f870e2c7700 1 -- 172.21.15.191:0/2877123276 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb1e0 con 0x5627b3513400 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- 172.21.15.191:0/2877123276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 msgr2=0x5627b37fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 --2- 172.21.15.191:0/2877123276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5627b4390450 tx=0x5627b385ede0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- 172.21.15.191:0/2877123276 shutdown_connections 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 --2- 172.21.15.191:0/2877123276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- 172.21.15.191:0/2877123276 >> 172.21.15.191:0/2877123276 conn(0x5627b3512c00 msgr2=0x5627b35ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- 172.21.15.191:0/2877123276 shutdown_connections 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- 172.21.15.191:0/2877123276 wait complete. 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.298+0000 7f8715b28a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b354e8c0 con 0x5627b3513400 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f87142d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f87142d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39006/0 (socket says 172.21.15.191:39006) 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 learned_addr learned my addr 172.21.15.191:0/4009619020 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353aea0 con 0x5627b3513400 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f87142d3700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5627b4391fb0 tx=0x5627b3603f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f870cac4700 1 -- 172.21.15.191:0/4009619020 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cba20 con 0x5627b3513400 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f870cac4700 1 -- 172.21.15.191:0/4009619020 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b35b3c80 con 0x5627b3513400 2024-04-27T04:34:44.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627b353a820 con 0x5627b3513400 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f870cac4700 1 -- 172.21.15.191:0/4009619020 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb600 con 0x5627b3513400 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.299+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627b353a9c0 con 0x5627b3513400 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f870cac4700 1 -- 172.21.15.191:0/4009619020 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5627b42b4dc0 con 0x5627b3513400 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f870cac4700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4c00 0x5627b37fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f870cac4700 1 -- 172.21.15.191:0/4009619020 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5627b358d340 con 0x5627b3513400 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f87132d1700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4c00 0x5627b37fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627b3513800 0x5627b37fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3513c00 con 0x5627b3513800 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.300+0000 7f87142d3700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627b3513800 0x5627b37fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f87132d1700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4c00 0x5627b37fb600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5627b37f5890 tx=0x5627b3603800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f87142d3700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627b3513800 0x5627b37fbb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 realm 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5000 con 0x5627b3513800 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5400 con 0x5627b3513800 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.301+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5800 con 0x5627b3513800 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627b38a5c00 0x5627b37fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5000 con 0x5627b38a5c00 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8713ad2700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627b38a5c00 0x5627b37fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8713ad2700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627b38a5c00 0x5627b37fc100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 20 searching for the correct realm 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.302+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5400 con 0x5627b38a5c00 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f8713ad2700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627b38a5400 0x5627b37fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5000 con 0x5627b38a5400 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f87132d1700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627b38a5400 0x5627b37fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f87132d1700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627b38a5400 0x5627b37fcc00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f87132d1700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627b38a5000 0x5627b37fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.303+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627b38a5000 0x5627b37fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627b38a5000 0x5627b37fd180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.304+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5400 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.305+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5400 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.306+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.307+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.308+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5400 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.309+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5400 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5c00 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.310+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5400 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3513c00 con 0x5627b3513800 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968400 con 0x5627b3513800 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.311+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968800 con 0x5627b38a5c00 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 zone default found 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 4 Realm: () 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 zonegroup default 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968c00 con 0x5627b38a5000 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969000 con 0x5627b38a5c00 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.312+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969400 con 0x5627b38a5c00 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.313+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.313+0000 7f8715b28a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.313+0000 7f8715b28a80 20 started sync module instance, tier type = 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.313+0000 7f8715b28a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.313+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969800 con 0x5627b38a5400 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.314+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.314+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94728515164928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968800 con 0x5627b38a5400 2024-04-27T04:34:44.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.314+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969000 con 0x5627b3513800 2024-04-27T04:34:44.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.315+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94728515164928] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.315+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.315+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94728515166080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969400 con 0x5627b3513800 2024-04-27T04:34:44.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.315+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968400 con 0x5627b38a5c00 2024-04-27T04:34:44.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.316+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94728515166080] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.316+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.316+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94728515167232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968c00 con 0x5627b38a5c00 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.316+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3513c00 con 0x5627b38a5400 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.317+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94728515167232] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.317+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.317+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94728515168384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b38a5800 con 0x5627b38a5400 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.317+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b3513800 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.318+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.318+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94728515169536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969800 con 0x5627b3513800 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.318+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968800 con 0x5627b38a5000 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.318+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94728515168384] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.319+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94728515169536] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.319+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.319+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94728515170688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969c00 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.319+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969000 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.322+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.322+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94728515171840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969400 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.322+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968000 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.322+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94728515170688] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94728515172992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969800 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94728515171840] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=0 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=1 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=2 2024-04-27T04:34:44.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=3 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=4 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=5 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.323+0000 7f8715b28a80 20 add_watcher() i=6 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.324+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94728515172992] v31'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.324+0000 7f8715b28a80 20 add_watcher() i=7 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.324+0000 7f8715b28a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.324+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968800 con 0x5627b3513800 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.324+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.324+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94728516755456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969c00 con 0x5627b3513800 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.325+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94728516755456] v31'74 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.325+0000 7f86faaa0700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.325+0000 7f8715b28a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.325+0000 7f8715b28a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f8715b28a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f8715b28a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b35b43c0 con 0x5627b3969400 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f87132d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f87132d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39016/0 (socket says 172.21.15.191:39016) 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f87132d1700 1 -- 172.21.15.191:0/2094274885 learned_addr learned my addr 172.21.15.191:0/2094274885 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f87132d1700 1 -- 172.21.15.191:0/2094274885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353ab60 con 0x5627b3969400 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f87132d1700 1 --2- 172.21.15.191:0/2094274885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5627b397e060 tx=0x5627b3603980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb19ed5d39f8f72c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f86fa29f700 1 -- 172.21.15.191:0/2094274885 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b35cb760 con 0x5627b3969400 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f86fa29f700 1 -- 172.21.15.191:0/2094274885 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b398a000 con 0x5627b3969400 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.326+0000 7f86fa29f700 1 -- 172.21.15.191:0/2094274885 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b398c000 con 0x5627b3969400 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 -- 172.21.15.191:0/2094274885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 msgr2=0x5627b37fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 --2- 172.21.15.191:0/2094274885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5627b397e060 tx=0x5627b3603980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 -- 172.21.15.191:0/2094274885 shutdown_connections 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 --2- 172.21.15.191:0/2094274885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 -- 172.21.15.191:0/2094274885 >> 172.21.15.191:0/2094274885 conn(0x5627b3969000 msgr2=0x5627b3600000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 -- 172.21.15.191:0/2094274885 shutdown_connections 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 -- 172.21.15.191:0/2094274885 wait complete. 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.327+0000 7f8715b28a80 1 Processor -- start 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f8715b28a80 1 -- start start 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f8715b28a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f8715b28a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b35b4500 con 0x5627b3969400 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f87132d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f87132d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39018/0 (socket says 172.21.15.191:39018) 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f87132d1700 1 -- 172.21.15.191:0/1432192607 learned_addr learned my addr 172.21.15.191:0/1432192607 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f87132d1700 1 -- 172.21.15.191:0/1432192607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b353ad00 con 0x5627b3969400 2024-04-27T04:34:44.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f87132d1700 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5627b397e2a0 tx=0x5627b36038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f86f8a9c700 1 -- 172.21.15.191:0/1432192607 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b398c160 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f86f8a9c700 1 -- 172.21.15.191:0/1432192607 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b398a180 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627b353a340 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f86f8a9c700 1 -- 172.21.15.191:0/1432192607 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b398c2c0 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.328+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627b353bd40 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.329+0000 7f86f8a9c700 1 -- 172.21.15.191:0/1432192607 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5627b42b5b80 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.329+0000 7f86f8a9c700 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b3a06000 0x5627b37fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.329+0000 7f86f8a9c700 1 -- 172.21.15.191:0/1432192607 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5627b358d500 con 0x5627b3969400 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.329+0000 7f8715b28a80 5 note: GC not initialized 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.329+0000 7f87142d3700 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b3a06000 0x5627b37fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.330+0000 7f87142d3700 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b3a06000 0x5627b37fd700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5627b38a1aa0 tx=0x5627b398d320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.347 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.330+0000 7f86f6297700 20 reqs_thread_entry: start 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.330+0000 7f8715b28a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.330+0000 7f8715b28a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.330+0000 7f86f4293700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f86f3a71700 20 reqs_thread_entry: start 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3969800 con 0x5627b3513800 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 20 rados_obj.operate() r=0 bl.length=459 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.331+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a06400 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v31'5 uv5 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f8715b28a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f8715b28a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f8715b28a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94728516757760 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a06800 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94728515169536 notify 133143986180 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5627b3903d40 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f870e2c7700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94728515169536 notifier 4364 bl.length()=533 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f870e2c7700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f870e2c7700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94728516757760 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.332+0000 7f870e2c7700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a76000 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.333+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== watch-notify(notify_complete (2) cookie 94728516757760 notify 133143986180 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5627b39cdba0 con 0x5627b3513800 2024-04-27T04:34:44.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.333+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.333+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a06c00 con 0x5627b38a5000 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v31'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f8715b28a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94728516757760 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a07000 con 0x5627b38a5c00 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94728515167232 notify 133143986177 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x5627b385dd40 con 0x5627b38a5c00 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f870e2c7700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986177 cookie 94728515167232 notifier 4364 bl.length()=170 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(67 notify.2 [notify cookie 94728516757760 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f870e2c7700 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a76400 con 0x5627b38a5c00 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.334+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== watch-notify(notify_complete (2) cookie 94728516757760 notify 133143986177 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5627b39cba00 con 0x5627b38a5c00 2024-04-27T04:34:44.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.335+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.351 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.352 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.353 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.335+0000 7f86faaa0700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.335+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94728516755456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a07400 con 0x5627b3513800 2024-04-27T04:34:44.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.336+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94728516755456] v31'75 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.336+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94728515164928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a07800 con 0x5627b38a5400 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.337+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94728515164928] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.337+0000 7f8715b28a80 20 remove_watcher() i=0 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.337+0000 7f8715b28a80 2 removed watcher, disabling cache 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.337+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94728515166080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a07c00 con 0x5627b3513800 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.338+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94728515166080] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.339+0000 7f8715b28a80 20 remove_watcher() i=1 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.339+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94728515167232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a07000 con 0x5627b38a5c00 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.339+0000 7f8713ad2700 1 -- 172.21.15.191:0/4009619020 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94728515167232] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951200 con 0x5627b38a5c00 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.339+0000 7f8715b28a80 20 remove_watcher() i=2 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.339+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94728515168384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a06400 con 0x5627b38a5400 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.341+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94728515168384] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3951440 con 0x5627b38a5400 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.341+0000 7f8715b28a80 20 remove_watcher() i=3 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.341+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94728515169536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a06800 con 0x5627b3513800 2024-04-27T04:34:44.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.342+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94728515169536] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b3513800 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.342+0000 7f8715b28a80 20 remove_watcher() i=4 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.342+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94728515170688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3a06c00 con 0x5627b38a5000 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.343+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94728515170688] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.343+0000 7f8715b28a80 20 remove_watcher() i=5 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.343+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94728515171840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968400 con 0x5627b38a5000 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.344+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94728515171840] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.344+0000 7f8715b28a80 20 remove_watcher() i=6 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.344+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94728515172992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627b3968c00 con 0x5627b38a5000 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f87142d3700 1 -- 172.21.15.191:0/4009619020 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94728515172992] v31'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b3950b40 con 0x5627b38a5000 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 20 remove_watcher() i=7 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627b38a5000 msgr2=0x5627b37fd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627b38a5000 0x5627b37fd180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627b3513800 msgr2=0x5627b37fbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627b3513800 0x5627b37fbb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627b38a5400 msgr2=0x5627b37fcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627b38a5400 0x5627b37fcc00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627b38a5c00 msgr2=0x5627b37fc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627b38a5c00 0x5627b37fc100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4c00 msgr2=0x5627b37fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4c00 0x5627b37fb600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5627b37f5890 tx=0x5627b3603800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 msgr2=0x5627b37fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/4009619020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3513400 0x5627b37fab00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5627b4391fb0 tx=0x5627b3603f20 comp rx=0 tx=0).stop 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 reap_dead start 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f87132d1700 1 -- 172.21.15.191:0/4009619020 reap_dead start 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 shutdown_connections 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 >> 172.21.15.191:0/4009619020 conn(0x5627b3512c00 msgr2=0x5627b35ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 shutdown_connections 2024-04-27T04:34:44.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/4009619020 wait complete. 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b3a06000 msgr2=0x5627b37fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b3a06000 0x5627b37fd700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5627b38a1aa0 tx=0x5627b398d320 comp rx=0 tx=0).stop 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 msgr2=0x5627b37fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5627b397e2a0 tx=0x5627b36038c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 shutdown_connections 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b3a06000 0x5627b37fd700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 --2- 172.21.15.191:0/1432192607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3969400 0x5627b37fc680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 >> 172.21.15.191:0/1432192607 conn(0x5627b3969000 msgr2=0x5627b3600000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 shutdown_connections 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.345+0000 7f8715b28a80 1 -- 172.21.15.191:0/1432192607 wait complete. 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4000 msgr2=0x5627b37fb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4000 0x5627b37fb080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5627b4390ab0 tx=0x5627b385e240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 msgr2=0x5627b37fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5627b42eba40 tx=0x5627b3603a40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 shutdown_connections 2024-04-27T04:34:44.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627b38a4000 0x5627b37fb080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 --2- 172.21.15.191:0/3258485068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627b3512800 0x5627b37fa580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 >> 172.21.15.191:0/3258485068 conn(0x5627b3512400 msgr2=0x5627b35ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 shutdown_connections 2024-04-27T04:34:44.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.346+0000 7f8715b28a80 1 -- 172.21.15.191:0/3258485068 wait complete. 2024-04-27T04:34:44.363 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.363 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-27T04:34:44.363 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-27T04:34:44.364 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-04-27T04:34:44.411 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.411 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.409+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.409+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.409+0000 7f3f29699a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x559ed91e0c00 0x559ed94c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.409+0000 7f3f29699a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559ed950b500 con 0x559ed91e0800 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.409+0000 7f3f29699a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x559ed9282a00 con 0x559ed91e0c00 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26e42700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x559ed91e0c00 0x559ed94c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f27643700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x559ed91e0800 0x559ed92cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55342/0 (socket says 172.21.15.191:55342) 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f27643700 1 -- 172.21.15.191:0/206196947 learned_addr learned my addr 172.21.15.191:0/206196947 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26641700 1 -- 172.21.15.191:0/206196947 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545855175 0 0) 0x559ed9fae000 con 0x559ed91e0800 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26641700 1 -- 172.21.15.191:0/206196947 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x559ed950b800 con 0x559ed91e0800 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26e42700 1 -- 172.21.15.191:0/206196947 >> v1:172.21.15.138:6789/0 conn(0x559ed91e0800 legacy=0x559ed92cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26e42700 1 -- 172.21.15.191:0/206196947 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9209d40 con 0x559ed91e0c00 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26e42700 1 --2- 172.21.15.191:0/206196947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0c00 0x559ed94c8580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x559ed94c3890 tx=0x559ed92d1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbfa88040a3eca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26641700 1 -- 172.21.15.191:0/206196947 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed92994a0 con 0x559ed91e0c00 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26641700 1 -- 172.21.15.191:0/206196947 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed950b500 con 0x559ed91e0c00 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.410+0000 7f3f26641700 1 -- 172.21.15.191:0/206196947 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9299600 con 0x559ed91e0c00 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 -- 172.21.15.191:0/206196947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0c00 msgr2=0x559ed94c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 --2- 172.21.15.191:0/206196947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0c00 0x559ed94c8580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x559ed94c3890 tx=0x559ed92d1980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 -- 172.21.15.191:0/206196947 shutdown_connections 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 --2- 172.21.15.191:0/206196947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0c00 0x559ed94c8580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 -- 172.21.15.191:0/206196947 >> 172.21.15.191:0/206196947 conn(0x559ed91e0400 msgr2=0x559ed92cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 -- 172.21.15.191:0/206196947 shutdown_connections 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 -- 172.21.15.191:0/206196947 wait complete. 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.411+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f29699a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f29699a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed9282500 con 0x559ed91e0800 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f27643700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f27643700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39050/0 (socket says 172.21.15.191:39050) 2024-04-27T04:34:44.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f27643700 1 -- 172.21.15.191:0/3855894043 learned_addr learned my addr 172.21.15.191:0/3855894043 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f27643700 1 -- 172.21.15.191:0/3855894043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9209ba0 con 0x559ed91e0800 2024-04-27T04:34:44.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f27643700 1 --2- 172.21.15.191:0/3855894043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x559ed9fb1770 tx=0x559ed92d1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78f65ccd08dcbad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f25e40700 1 -- 172.21.15.191:0/3855894043 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed92991e0 con 0x559ed91e0800 2024-04-27T04:34:44.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f25e40700 1 -- 172.21.15.191:0/3855894043 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed950bb00 con 0x559ed91e0800 2024-04-27T04:34:44.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.412+0000 7f3f25e40700 1 -- 172.21.15.191:0/3855894043 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed92994a0 con 0x559ed91e0800 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- 172.21.15.191:0/3855894043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 msgr2=0x559ed94c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3855894043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x559ed9fb1770 tx=0x559ed92d1b60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- 172.21.15.191:0/3855894043 shutdown_connections 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3855894043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- 172.21.15.191:0/3855894043 >> 172.21.15.191:0/3855894043 conn(0x559ed91e0400 msgr2=0x559ed92cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- 172.21.15.191:0/3855894043 shutdown_connections 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- 172.21.15.191:0/3855894043 wait complete. 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f29699a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed92823c0 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.413+0000 7f3f27643700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f27643700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39054/0 (socket says 172.21.15.191:39054) 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f27643700 1 -- 172.21.15.191:0/1382250280 learned_addr learned my addr 172.21.15.191:0/1382250280 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f27643700 1 -- 172.21.15.191:0/1382250280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9209a00 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f27643700 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x559ed9fb1e30 tx=0x559ed92d1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f2463d700 1 -- 172.21.15.191:0/1382250280 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9299600 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f2463d700 1 -- 172.21.15.191:0/1382250280 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed9281c80 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f2463d700 1 -- 172.21.15.191:0/1382250280 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed92991e0 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ed9209860 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.414+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ed92096c0 con 0x559ed91e0800 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f2463d700 1 -- 172.21.15.191:0/1382250280 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559ed9f82b00 con 0x559ed91e0800 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f29699a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f29699a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed921c640 con 0x559ed91e1400 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f2463d700 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956e000 0x559ed94c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f2463d700 1 -- 172.21.15.191:0/1382250280 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x559ed925b180 con 0x559ed91e0800 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27e44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27e44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39068/0 (socket says 172.21.15.191:39068) 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27e44700 1 -- 172.21.15.191:0/1277968350 learned_addr learned my addr 172.21.15.191:0/1277968350 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27643700 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956e000 0x559ed94c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27e44700 1 -- 172.21.15.191:0/1277968350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9208000 con 0x559ed91e1400 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27643700 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956e000 0x559ed94c9080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x559eda06af00 tx=0x559ed952a240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.415+0000 7f3f27e44700 1 --2- 172.21.15.191:0/1277968350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559ed9fb1e90 tx=0x559ed952ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69212ab25c71632e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f21e38700 1 -- 172.21.15.191:0/1277968350 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9299600 con 0x559ed91e1400 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f21e38700 1 -- 172.21.15.191:0/1277968350 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed950a480 con 0x559ed91e1400 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f21e38700 1 -- 172.21.15.191:0/1277968350 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed92991e0 con 0x559ed91e1400 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- 172.21.15.191:0/1277968350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 msgr2=0x559ed94c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 --2- 172.21.15.191:0/1277968350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559ed9fb1e90 tx=0x559ed952ade0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- 172.21.15.191:0/1277968350 shutdown_connections 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 --2- 172.21.15.191:0/1277968350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- 172.21.15.191:0/1277968350 >> 172.21.15.191:0/1277968350 conn(0x559ed91e0c00 msgr2=0x559ed92cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- 172.21.15.191:0/1277968350 shutdown_connections 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- 172.21.15.191:0/1277968350 wait complete. 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.416+0000 7f3f29699a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed921c8c0 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f27e44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f27e44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39076/0 (socket says 172.21.15.191:39076) 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 learned_addr learned my addr 172.21.15.191:0/3318176734 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9208ea0 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f27e44700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559eda06a450 tx=0x559ed92d1f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f20635700 1 -- 172.21.15.191:0/3318176734 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9299a20 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f20635700 1 -- 172.21.15.191:0/3318176734 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed950a180 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f20635700 1 -- 172.21.15.191:0/3318176734 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9299600 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ed9208820 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.417+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ed92089c0 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f20635700 1 -- 172.21.15.191:0/3318176734 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559ed9f82dc0 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f20635700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956ec00 0x559ed94c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f20635700 1 -- 172.21.15.191:0/3318176734 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x559ed925b340 con 0x559ed91e1400 2024-04-27T04:34:44.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f26e42700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956ec00 0x559ed94c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f26e42700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956ec00 0x559ed94c9600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x559ed94c3890 tx=0x559ed92d1800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.418+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559ed91e1800 0x559ed94c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed91e1c00 con 0x559ed91e1800 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f27e44700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559ed91e1800 0x559ed94c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f27e44700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559ed91e1800 0x559ed94c9b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 realm 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f000 con 0x559ed91e1800 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f400 con 0x559ed91e1800 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.419+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f800 con 0x559ed91e1800 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x559ed956fc00 0x559ed94ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f000 con 0x559ed956fc00 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f27643700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x559ed956fc00 0x559ed94ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f27643700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x559ed956fc00 0x559ed94ca100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 20 searching for the correct realm 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.420+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f400 con 0x559ed956fc00 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f27643700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x559ed956f400 0x559ed94cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f000 con 0x559ed956f400 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f26e42700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x559ed956f400 0x559ed94cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f26e42700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x559ed956f400 0x559ed94cac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f26e42700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x559ed956f000 0x559ed94cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.421+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.422+0000 7f3f27e44700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x559ed956f000 0x559ed94cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.422+0000 7f3f27e44700 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x559ed956f000 0x559ed94cb180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.422+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.422+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.422+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.422+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f400 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.423+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f400 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.424+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.425+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f400 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.426+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.427+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f400 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956fc00 2024-04-27T04:34:44.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.428+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f400 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed91e1c00 con 0x559ed91e1800 2024-04-27T04:34:44.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632400 con 0x559ed91e1800 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632800 con 0x559ed956fc00 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 zone default found 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 4 Realm: () 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 zonegroup default 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.429+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632c00 con 0x559ed956f000 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633000 con 0x559ed956fc00 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633400 con 0x559ed956fc00 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 started sync module instance, tier type = 2024-04-27T04:34:44.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.430+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633800 con 0x559ed956f400 2024-04-27T04:34:44.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.431+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.431+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94140738829056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632800 con 0x559ed956f400 2024-04-27T04:34:44.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.431+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633000 con 0x559ed91e1800 2024-04-27T04:34:44.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.432+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.432+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94140738830208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633400 con 0x559ed91e1800 2024-04-27T04:34:44.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.432+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632400 con 0x559ed956fc00 2024-04-27T04:34:44.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.434+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94140738830208] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.434+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94140738829056] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.434+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.434+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94140738831360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632c00 con 0x559ed956fc00 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.434+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed91e1c00 con 0x559ed956f400 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94140738832512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed956f800 con 0x559ed956f400 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed91e1800 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94140738831360] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94140738833664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633800 con 0x559ed91e1800 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.435+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632800 con 0x559ed956f000 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.436+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94140738832512] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.436+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94140738833664] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.436+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.436+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94140738834816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633c00 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.436+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633000 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.437+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94140738834816] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.438+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.438+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94140738835968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633400 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.438+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632000 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94140738837120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633800 con 0x559ed956f000 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=0 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=1 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=2 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=3 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=4 2024-04-27T04:34:44.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=5 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94140738835968] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.439+0000 7f3f29699a80 20 add_watcher() i=6 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.440+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94140738837120] v31'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.440+0000 7f3f29699a80 20 add_watcher() i=7 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.440+0000 7f3f29699a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.440+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632800 con 0x559ed91e1800 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.440+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.440+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94140740395008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633c00 con 0x559ed91e1800 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94140740395008] v31'76 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f0e611700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f29699a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f29699a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f29699a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.441+0000 7f3f29699a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed92823c0 con 0x559ed9633400 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f26e42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f26e42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39078/0 (socket says 172.21.15.191:39078) 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f26e42700 1 -- 172.21.15.191:0/3565899042 learned_addr learned my addr 172.21.15.191:0/3565899042 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f26e42700 1 -- 172.21.15.191:0/3565899042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9208b60 con 0x559ed9633400 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f26e42700 1 --2- 172.21.15.191:0/3565899042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x559ed95d5800 tx=0x559ed92d19e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415c3f931933e322 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f0de10700 1 -- 172.21.15.191:0/3565899042 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9299760 con 0x559ed9633400 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f0de10700 1 -- 172.21.15.191:0/3565899042 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed9648000 con 0x559ed9633400 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.442+0000 7f3f0de10700 1 -- 172.21.15.191:0/3565899042 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9658000 con 0x559ed9633400 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- 172.21.15.191:0/3565899042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 msgr2=0x559ed94ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3565899042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x559ed95d5800 tx=0x559ed92d19e0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- 172.21.15.191:0/3565899042 shutdown_connections 2024-04-27T04:34:44.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3565899042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- 172.21.15.191:0/3565899042 >> 172.21.15.191:0/3565899042 conn(0x559ed9633000 msgr2=0x559ed92ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- 172.21.15.191:0/3565899042 shutdown_connections 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- 172.21.15.191:0/3565899042 wait complete. 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 Processor -- start 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- start start 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f29699a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed9282500 con 0x559ed9633400 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f26e42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f26e42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39088/0 (socket says 172.21.15.191:39088) 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.443+0000 7f3f26e42700 1 -- 172.21.15.191:0/978576449 learned_addr learned my addr 172.21.15.191:0/978576449 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f26e42700 1 -- 172.21.15.191:0/978576449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed9208d00 con 0x559ed9633400 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f26e42700 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x559ed9647260 tx=0x559ed92d18c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f0c60d700 1 -- 172.21.15.191:0/978576449 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed9658160 con 0x559ed9633400 2024-04-27T04:34:44.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f0c60d700 1 -- 172.21.15.191:0/978576449 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ed9648180 con 0x559ed9633400 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ed9208340 con 0x559ed9633400 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f0c60d700 1 -- 172.21.15.191:0/978576449 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559ed96582c0 con 0x559ed9633400 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.444+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ed9209d40 con 0x559ed9633400 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.445+0000 7f3f0c60d700 1 -- 172.21.15.191:0/978576449 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559ed9f83b80 con 0x559ed9633400 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.445+0000 7f3f0c60d700 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed96d2000 0x559ed94cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.445+0000 7f3f0c60d700 1 -- 172.21.15.191:0/978576449 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x559ed925b500 con 0x559ed9633400 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.445+0000 7f3f27e44700 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed96d2000 0x559ed94cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.445+0000 7f3f29699a80 5 note: GC not initialized 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.445+0000 7f3f27e44700 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed96d2000 0x559ed94cb700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x559ed95d5320 tx=0x559ed9657260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f09e08700 20 reqs_thread_entry: start 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f29699a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f29699a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f07e04700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f075e2700 20 reqs_thread_entry: start 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f29699a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.446+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633800 con 0x559ed91e1800 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632800 con 0x559ed91e1800 2024-04-27T04:34:44.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.447+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d2800 con 0x559ed91e1800 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.448+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~513] v31'6 uv6 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.448+0000 7f3f29699a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.448+0000 7f3f29699a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.448+0000 7f3f29699a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.448+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94140740397312 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d2c00 con 0x559ed91e1800 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== watch-notify(notify (1) cookie 94140738833664 notify 133143986181 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x559ed95d7d40 con 0x559ed91e1800 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(65 notify.4 [notify cookie 94140740397312 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f21637700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94140738833664 notifier 4372 bl.length()=689 2024-04-27T04:34:44.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f21637700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f21637700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f21637700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9746000 con 0x559ed91e1800 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94140740397312 notify 133143986181 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559ed969bba0 con 0x559ed91e1800 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.449+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d3000 con 0x559ed91e1800 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f29699a80 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f29699a80 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f29699a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94140740397312 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d3400 con 0x559ed956f400 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== watch-notify(notify (1) cookie 94140738829056 notify 133143986178 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x559ed9699a00 con 0x559ed956f400 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f21e38700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94140738829056 notifier 4372 bl.length()=166 2024-04-27T04:34:44.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f21e38700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f21e38700 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(68 notify.0 [notify cookie 94140740397312 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.450+0000 7f3f21e38700 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633c00 con 0x559ed956f400 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.451+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.451+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== watch-notify(notify_complete (2) cookie 94140740397312 notify 133143986178 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559ed9697040 con 0x559ed956f400 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.470 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo1", 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "jz86b7ja0hhbWeeOq0D9YBdf4DZGkjxjUCDHfzCA" 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.471 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.472 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.473 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.451+0000 7f3f0e611700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.451+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94140740395008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d3800 con 0x559ed91e1800 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.452+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94140740395008] v31'77 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.452+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94140738829056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d3c00 con 0x559ed956f400 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.453+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94140738829056] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.453+0000 7f3f29699a80 20 remove_watcher() i=0 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.453+0000 7f3f29699a80 2 removed watcher, disabling cache 2024-04-27T04:34:44.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.453+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94140738830208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d3400 con 0x559ed91e1800 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.454+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94140738830208] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.454+0000 7f3f29699a80 20 remove_watcher() i=1 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.454+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94140738831360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d3c00 con 0x559ed956fc00 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.455+0000 7f3f27643700 1 -- 172.21.15.191:0/3318176734 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94140738831360] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b440 con 0x559ed956fc00 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.455+0000 7f3f29699a80 20 remove_watcher() i=2 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.455+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94140738832512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d2800 con 0x559ed956f400 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.456+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94140738832512] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962b680 con 0x559ed956f400 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.456+0000 7f3f29699a80 20 remove_watcher() i=3 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.456+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94140738833664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d2400 con 0x559ed91e1800 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.457+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94140738833664] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed91e1800 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.457+0000 7f3f29699a80 20 remove_watcher() i=4 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.457+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94140738834816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed96d2c00 con 0x559ed956f000 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.458+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94140738834816] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.458+0000 7f3f29699a80 20 remove_watcher() i=5 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.458+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94140738835968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9633c00 con 0x559ed956f000 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.459+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94140738835968] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.459+0000 7f3f29699a80 20 remove_watcher() i=6 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.459+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94140738837120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559ed9632800 con 0x559ed956f000 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f27e44700 1 -- 172.21.15.191:0/3318176734 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94140738837120] v31'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559ed962ad80 con 0x559ed956f000 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 20 remove_watcher() i=7 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x559ed956f000 msgr2=0x559ed94cb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x559ed956f000 0x559ed94cb180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559ed91e1800 msgr2=0x559ed94c9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559ed91e1800 0x559ed94c9b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x559ed956f400 msgr2=0x559ed94cac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x559ed956f400 0x559ed94cac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x559ed956fc00 msgr2=0x559ed94ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x559ed956fc00 0x559ed94ca100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956ec00 msgr2=0x559ed94c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956ec00 0x559ed94c9600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x559ed94c3890 tx=0x559ed92d1800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 msgr2=0x559ed94c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 --2- 172.21.15.191:0/3318176734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e1400 0x559ed94c8b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559eda06a450 tx=0x559ed92d1f20 comp rx=0 tx=0).stop 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 reap_dead start 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f26e42700 1 -- 172.21.15.191:0/3318176734 reap_dead start 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 shutdown_connections 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 >> 172.21.15.191:0/3318176734 conn(0x559ed91e0c00 msgr2=0x559ed92cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 shutdown_connections 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.460+0000 7f3f29699a80 1 -- 172.21.15.191:0/3318176734 wait complete. 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed96d2000 msgr2=0x559ed94cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed96d2000 0x559ed94cb700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x559ed95d5320 tx=0x559ed9657260 comp rx=0 tx=0).stop 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 msgr2=0x559ed94ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x559ed9647260 tx=0x559ed92d18c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 shutdown_connections 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed96d2000 0x559ed94cb700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/978576449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed9633400 0x559ed94ca680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 >> 172.21.15.191:0/978576449 conn(0x559ed9633000 msgr2=0x559ed92ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 shutdown_connections 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/978576449 wait complete. 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956e000 msgr2=0x559ed94c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956e000 0x559ed94c9080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x559eda06af00 tx=0x559ed952a240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 msgr2=0x559ed94c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x559ed9fb1e30 tx=0x559ed92d1920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 shutdown_connections 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x559ed956e000 0x559ed94c9080 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 --2- 172.21.15.191:0/1382250280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ed91e0800 0x559ed94c8580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 >> 172.21.15.191:0/1382250280 conn(0x559ed91e0400 msgr2=0x559ed92cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 shutdown_connections 2024-04-27T04:34:44.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.461+0000 7f3f29699a80 1 -- 172.21.15.191:0/1382250280 wait complete. 2024-04-27T04:34:44.483 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'jz86b7ja0hhbWeeOq0D9YBdf4DZGkjxjUCDHfzCA'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.483 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-27T04:34:44.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-27T04:34:44.483 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-04-27T04:34:44.533 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.533 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea36195a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55837ae20c00 0x55837b108580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea36195a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55837b14b500 con 0x55837ae20800 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea36195a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55837aec2a00 con 0x55837ae20c00 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea3393e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55837ae20c00 0x55837b108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea3413f700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55837ae20800 0x55837af0c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55356/0 (socket says 172.21.15.191:55356) 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea3393e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55837ae20c00 0x55837b108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39090/0 (socket says 172.21.15.191:39090) 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.531+0000 7fea3413f700 1 -- 172.21.15.191:0/807354538 learned_addr learned my addr 172.21.15.191:0/807354538 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea3393e700 1 -- 172.21.15.191:0/807354538 >> v1:172.21.15.138:6789/0 conn(0x55837ae20800 legacy=0x55837af0c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea3393e700 1 -- 172.21.15.191:0/807354538 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae49d40 con 0x55837ae20c00 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea3393e700 1 --2- 172.21.15.191:0/807354538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20c00 0x55837b108580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55837b103890 tx=0x55837af11920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5593394392dd7663 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea3313d700 1 -- 172.21.15.191:0/807354538 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed94a0 con 0x55837ae20c00 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea3313d700 1 -- 172.21.15.191:0/807354538 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837b14b800 con 0x55837ae20c00 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea3313d700 1 -- 172.21.15.191:0/807354538 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed9600 con 0x55837ae20c00 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 -- 172.21.15.191:0/807354538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20c00 msgr2=0x55837b108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 --2- 172.21.15.191:0/807354538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20c00 0x55837b108580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55837b103890 tx=0x55837af11920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 -- 172.21.15.191:0/807354538 shutdown_connections 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 --2- 172.21.15.191:0/807354538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20c00 0x55837b108580 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 -- 172.21.15.191:0/807354538 >> 172.21.15.191:0/807354538 conn(0x55837ae20400 msgr2=0x55837af0c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 -- 172.21.15.191:0/807354538 shutdown_connections 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.532+0000 7fea36195a80 1 -- 172.21.15.191:0/807354538 wait complete. 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea36195a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea36195a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837aec2500 con 0x55837ae20800 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea3413f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea3413f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39106/0 (socket says 172.21.15.191:39106) 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.533+0000 7fea3413f700 1 -- 172.21.15.191:0/2695218632 learned_addr learned my addr 172.21.15.191:0/2695218632 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea3413f700 1 -- 172.21.15.191:0/2695218632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae49ba0 con 0x55837ae20800 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea3413f700 1 --2- 172.21.15.191:0/2695218632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55837bc8b8c0 tx=0x55837af11b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d31c966aa7a118 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea3293c700 1 -- 172.21.15.191:0/2695218632 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed94a0 con 0x55837ae20800 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea3293c700 1 -- 172.21.15.191:0/2695218632 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837b14bb00 con 0x55837ae20800 2024-04-27T04:34:44.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea3293c700 1 -- 172.21.15.191:0/2695218632 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed9600 con 0x55837ae20800 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 -- 172.21.15.191:0/2695218632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 msgr2=0x55837b108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 --2- 172.21.15.191:0/2695218632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55837bc8b8c0 tx=0x55837af11b60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 -- 172.21.15.191:0/2695218632 shutdown_connections 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 --2- 172.21.15.191:0/2695218632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 -- 172.21.15.191:0/2695218632 >> 172.21.15.191:0/2695218632 conn(0x55837ae20400 msgr2=0x55837af0d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 -- 172.21.15.191:0/2695218632 shutdown_connections 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 -- 172.21.15.191:0/2695218632 wait complete. 2024-04-27T04:34:44.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.534+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea36195a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea36195a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837aec23c0 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea3413f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea3413f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39122/0 (socket says 172.21.15.191:39122) 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea3413f700 1 -- 172.21.15.191:0/1136404196 learned_addr learned my addr 172.21.15.191:0/1136404196 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea3413f700 1 -- 172.21.15.191:0/1136404196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae49a00 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea3413f700 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55837bc8baa0 tx=0x55837af11a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea31139700 1 -- 172.21.15.191:0/1136404196 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed98c0 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea31139700 1 -- 172.21.15.191:0/1136404196 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837aec1c80 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea31139700 1 -- 172.21.15.191:0/1136404196 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed94a0 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55837ae49860 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.535+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55837ae496c0 con 0x55837ae20800 2024-04-27T04:34:44.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea31139700 1 -- 172.21.15.191:0/1136404196 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55837bbc2b00 con 0x55837ae20800 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea36195a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea36195a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ae5c640 con 0x55837ae21400 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea31139700 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4000 0x55837b109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea31139700 1 -- 172.21.15.191:0/1136404196 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55837ae9b180 con 0x55837ae20800 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea3413f700 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4000 0x55837b109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea34940700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea34940700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39134/0 (socket says 172.21.15.191:39134) 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.536+0000 7fea34940700 1 -- 172.21.15.191:0/228308595 learned_addr learned my addr 172.21.15.191:0/228308595 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea34940700 1 -- 172.21.15.191:0/228308595 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae48000 con 0x55837ae21400 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea3413f700 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4000 0x55837b109080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55837bc9f020 tx=0x55837b16c240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea34940700 1 --2- 172.21.15.191:0/228308595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55837bc9e3c0 tx=0x55837b16d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29605b0f4473ad13 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea2e934700 1 -- 172.21.15.191:0/228308595 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed98c0 con 0x55837ae21400 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea2e934700 1 -- 172.21.15.191:0/228308595 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837b14a300 con 0x55837ae21400 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea2e934700 1 -- 172.21.15.191:0/228308595 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed94a0 con 0x55837ae21400 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 -- 172.21.15.191:0/228308595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 msgr2=0x55837b108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 --2- 172.21.15.191:0/228308595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55837bc9e3c0 tx=0x55837b16d0e0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 -- 172.21.15.191:0/228308595 shutdown_connections 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 --2- 172.21.15.191:0/228308595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 -- 172.21.15.191:0/228308595 >> 172.21.15.191:0/228308595 conn(0x55837ae20c00 msgr2=0x55837af0d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 -- 172.21.15.191:0/228308595 shutdown_connections 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.537+0000 7fea36195a80 1 -- 172.21.15.191:0/228308595 wait complete. 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea36195a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea36195a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ae5c8c0 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea34940700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea34940700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39148/0 (socket says 172.21.15.191:39148) 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 learned_addr learned my addr 172.21.15.191:0/4093953901 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae48ea0 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea34940700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55837bc9e210 tx=0x55837b16ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea2d131700 1 -- 172.21.15.191:0/4093953901 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed9b80 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea2d131700 1 -- 172.21.15.191:0/4093953901 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837aec1c80 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55837ae48820 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.538+0000 7fea2d131700 1 -- 172.21.15.191:0/4093953901 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed98c0 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.539+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55837ae489c0 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.539+0000 7fea2d131700 1 -- 172.21.15.191:0/4093953901 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55837bbc2dc0 con 0x55837ae21400 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.539+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.539+0000 7fea2d131700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4c00 0x55837b109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea2d131700 1 -- 172.21.15.191:0/4093953901 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55837ae9b340 con 0x55837ae21400 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea3393e700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4c00 0x55837b109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55837ae21800 0x55837b109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837ae21c00 con 0x55837ae21800 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea34940700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55837ae21800 0x55837b109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea3393e700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4c00 0x55837b109600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55837b103890 tx=0x55837af11800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea34940700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55837ae21800 0x55837b109b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea36195a80 20 realm 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.540+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5000 con 0x55837ae21800 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5400 con 0x55837ae21800 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5800 con 0x55837ae21800 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55837b1b5c00 0x55837b10a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5000 con 0x55837b1b5c00 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea3413f700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55837b1b5c00 0x55837b10a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.541+0000 7fea3413f700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55837b1b5c00 0x55837b10a100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea36195a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea36195a80 20 searching for the correct realm 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5400 con 0x55837b1b5c00 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea3413f700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55837b1b5400 0x55837b10ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5000 con 0x55837b1b5400 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea3393e700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55837b1b5400 0x55837b10ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.542+0000 7fea3393e700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55837b1b5400 0x55837b10ac00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.543+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.543+0000 7fea3393e700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55837b1b5000 0x55837b10b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.543+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.543+0000 7fea34940700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55837b1b5000 0x55837b10b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.543+0000 7fea34940700 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55837b1b5000 0x55837b10b180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.543+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.544+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5400 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.545+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5400 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.546+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.547+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5400 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.548+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5400 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5c00 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.549+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5400 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837ae21c00 con 0x55837ae21800 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276400 con 0x55837ae21800 2024-04-27T04:34:44.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.550+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276800 con 0x55837b1b5c00 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 zone default found 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 4 Realm: () 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 zonegroup default 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276c00 con 0x55837b1b5000 2024-04-27T04:34:44.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277000 con 0x55837b1b5c00 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277400 con 0x55837b1b5c00 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.551+0000 7fea36195a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.552+0000 7fea36195a80 20 started sync module instance, tier type = 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.552+0000 7fea36195a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.552+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277800 con 0x55837b1b5400 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.553+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.553+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94023193721600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276800 con 0x55837b1b5400 2024-04-27T04:34:44.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.553+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277000 con 0x55837ae21800 2024-04-27T04:34:44.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.553+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94023193721600] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.554+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.554+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94023193722752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277400 con 0x55837ae21800 2024-04-27T04:34:44.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.554+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276400 con 0x55837b1b5c00 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.555+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.555+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94023193723904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276c00 con 0x55837b1b5c00 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.555+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837ae21c00 con 0x55837b1b5400 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.555+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94023193722752] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.555+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94023193723904] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.556+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.556+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94023193725056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b1b5800 con 0x55837b1b5400 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.556+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837ae21800 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94023193726208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277800 con 0x55837ae21800 2024-04-27T04:34:44.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276800 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94023193725056] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94023193727360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277c00 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277000 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.557+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94023193726208] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.558+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94023193727360] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.558+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.558+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94023193728512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277400 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.558+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276000 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94023193729664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277800 con 0x55837b1b5000 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=0 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=1 2024-04-27T04:34:44.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=2 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=3 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=4 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=5 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94023193728512] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.559+0000 7fea36195a80 20 add_watcher() i=6 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.560+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94023193729664] v31'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.560+0000 7fea36195a80 20 add_watcher() i=7 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.560+0000 7fea36195a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.560+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b276800 con 0x55837ae21800 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.560+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.560+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94023195312128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277c00 con 0x55837ae21800 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.561+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94023195312128] v31'78 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.561+0000 7fea1b10d700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.561+0000 7fea36195a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.561+0000 7fea36195a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea36195a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea36195a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837aec23c0 con 0x55837b277400 2024-04-27T04:34:44.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea3393e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea3393e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39160/0 (socket says 172.21.15.191:39160) 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea3393e700 1 -- 172.21.15.191:0/427720052 learned_addr learned my addr 172.21.15.191:0/427720052 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea3393e700 1 -- 172.21.15.191:0/427720052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae48b60 con 0x55837b277400 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea3393e700 1 --2- 172.21.15.191:0/427720052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55837b28c060 tx=0x55837af11980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e27b6fcca1f1f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea1a90c700 1 -- 172.21.15.191:0/427720052 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837aed9760 con 0x55837b277400 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea1a90c700 1 -- 172.21.15.191:0/427720052 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837b298000 con 0x55837b277400 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.562+0000 7fea1a90c700 1 -- 172.21.15.191:0/427720052 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837b29a000 con 0x55837b277400 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 -- 172.21.15.191:0/427720052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 msgr2=0x55837b10a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 --2- 172.21.15.191:0/427720052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55837b28c060 tx=0x55837af11980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 -- 172.21.15.191:0/427720052 shutdown_connections 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 --2- 172.21.15.191:0/427720052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 -- 172.21.15.191:0/427720052 >> 172.21.15.191:0/427720052 conn(0x55837b277000 msgr2=0x55837af0e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 -- 172.21.15.191:0/427720052 shutdown_connections 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 -- 172.21.15.191:0/427720052 wait complete. 2024-04-27T04:34:44.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.563+0000 7fea36195a80 1 Processor -- start 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea36195a80 1 -- start start 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea36195a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea36195a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837aec2500 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea3393e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea3393e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39162/0 (socket says 172.21.15.191:39162) 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea3393e700 1 -- 172.21.15.191:0/262555345 learned_addr learned my addr 172.21.15.191:0/262555345 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea3393e700 1 -- 172.21.15.191:0/262555345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ae48d00 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea3393e700 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55837b28c270 tx=0x55837af118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea19109700 1 -- 172.21.15.191:0/262555345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837b29a160 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea19109700 1 -- 172.21.15.191:0/262555345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55837b298180 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55837ae48340 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea19109700 1 -- 172.21.15.191:0/262555345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55837b29a2c0 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.564+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55837ae49d40 con 0x55837b277400 2024-04-27T04:34:44.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.565+0000 7fea19109700 1 -- 172.21.15.191:0/262555345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55837bbc3b80 con 0x55837b277400 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.565+0000 7fea19109700 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b312000 0x55837b10b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.565+0000 7fea19109700 1 -- 172.21.15.191:0/262555345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55837ae9b500 con 0x55837b277400 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.565+0000 7fea34940700 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b312000 0x55837b10b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.565+0000 7fea36195a80 5 note: GC not initialized 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.566+0000 7fea34940700 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b312000 0x55837b10b700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55837b1b19b0 tx=0x55837b297260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.566+0000 7fea16904700 20 reqs_thread_entry: start 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.566+0000 7fea36195a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.566+0000 7fea36195a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.566+0000 7fea14900700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.566+0000 7fea140de700 20 reqs_thread_entry: start 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b277800 con 0x55837ae21800 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv6 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.567+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b312400 con 0x55837ae21800 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.568+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v31'7 uv7 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.568+0000 7fea36195a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.568+0000 7fea36195a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.568+0000 7fea36195a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.568+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94023195314432 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b312800 con 0x55837ae21800 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94023193726208 notify 133143986182 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x55837b21dd40 con 0x55837ae21800 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea2e934700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986182 cookie 94023193726208 notifier 4380 bl.length()=689 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea2e934700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea2e934700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94023195314432 out=8b] v0'0 uv57 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea2e934700 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b384000 con 0x55837ae21800 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.569+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94023195314432 notify 133143986182 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55837b2ddba0 con 0x55837ae21800 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.590 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.591 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo1", 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.592 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.593 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.570+0000 7fea1b10d700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.570+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94023195312128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b313400 con 0x55837ae21800 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.571+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94023195312128] v31'79 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.571+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94023193721600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b313800 con 0x55837b1b5400 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.571+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94023193721600] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.572+0000 7fea36195a80 20 remove_watcher() i=0 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.572+0000 7fea36195a80 2 removed watcher, disabling cache 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.572+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94023193722752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b313c00 con 0x55837ae21800 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.573+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94023193722752] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.573+0000 7fea36195a80 20 remove_watcher() i=1 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.573+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94023193723904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b384000 con 0x55837b1b5c00 2024-04-27T04:34:44.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.574+0000 7fea3413f700 1 -- 172.21.15.191:0/4093953901 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94023193723904] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267200 con 0x55837b1b5c00 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.574+0000 7fea36195a80 20 remove_watcher() i=2 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.574+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94023193725056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b384800 con 0x55837b1b5400 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.575+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94023193725056] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b267440 con 0x55837b1b5400 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.575+0000 7fea36195a80 20 remove_watcher() i=3 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.575+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94023193726208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b384c00 con 0x55837ae21800 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.576+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94023193726208] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837ae21800 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.576+0000 7fea36195a80 20 remove_watcher() i=4 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.576+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94023193727360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b385000 con 0x55837b1b5000 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.577+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94023193727360] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.577+0000 7fea36195a80 20 remove_watcher() i=5 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.577+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94023193728512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b312400 con 0x55837b1b5000 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.578+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94023193728512] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.578+0000 7fea36195a80 20 remove_watcher() i=6 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.578+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94023193729664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55837b312800 con 0x55837b1b5000 2024-04-27T04:34:44.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.578+0000 7fea34940700 1 -- 172.21.15.191:0/4093953901 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94023193729664] v31'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55837b266b40 con 0x55837b1b5000 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.578+0000 7fea36195a80 20 remove_watcher() i=7 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55837b1b5000 msgr2=0x55837b10b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55837b1b5000 0x55837b10b180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55837ae21800 msgr2=0x55837b109b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55837ae21800 0x55837b109b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55837b1b5400 msgr2=0x55837b10ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55837b1b5400 0x55837b10ac00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55837b1b5c00 msgr2=0x55837b10a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55837b1b5c00 0x55837b10a100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4c00 msgr2=0x55837b109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4c00 0x55837b109600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55837b103890 tx=0x55837af11800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 msgr2=0x55837b108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/4093953901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae21400 0x55837b108b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55837bc9e210 tx=0x55837b16ccc0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 reap_dead start 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea3393e700 1 -- 172.21.15.191:0/4093953901 reap_dead start 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 shutdown_connections 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 >> 172.21.15.191:0/4093953901 conn(0x55837ae20c00 msgr2=0x55837af0d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 shutdown_connections 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/4093953901 wait complete. 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b312000 msgr2=0x55837b10b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b312000 0x55837b10b700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55837b1b19b0 tx=0x55837b297260 comp rx=0 tx=0).stop 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 msgr2=0x55837b10a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55837b28c270 tx=0x55837af118c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 shutdown_connections 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b312000 0x55837b10b700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 --2- 172.21.15.191:0/262555345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837b277400 0x55837b10a680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 >> 172.21.15.191:0/262555345 conn(0x55837b277000 msgr2=0x55837af0e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 shutdown_connections 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.579+0000 7fea36195a80 1 -- 172.21.15.191:0/262555345 wait complete. 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4000 msgr2=0x55837b109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4000 0x55837b109080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55837bc9f020 tx=0x55837b16c240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 msgr2=0x55837b108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55837bc8baa0 tx=0x55837af11a40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 shutdown_connections 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55837b1b4000 0x55837b109080 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 --2- 172.21.15.191:0/1136404196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55837ae20800 0x55837b108580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 >> 172.21.15.191:0/1136404196 conn(0x55837ae20400 msgr2=0x55837af0d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 shutdown_connections 2024-04-27T04:34:44.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.580+0000 7fea36195a80 1 -- 172.21.15.191:0/1136404196 wait complete. 2024-04-27T04:34:44.603 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.604 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.604 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.604 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.647+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.647+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.647+0000 7fa28a9b3a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.647+0000 7fa28a9b3a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562b6a893500 con 0x562b6a568c00 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.647+0000 7fa28a9b3a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x562b6a60aa00 con 0x562b6a568800 2024-04-27T04:34:44.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28895d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28815c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x562b6a568c00 0x562b6a654800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55364/0 (socket says 172.21.15.191:55364) 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28815c700 1 -- 172.21.15.191:0/614793777 learned_addr learned my addr 172.21.15.191:0/614793777 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28795b700 1 -- 172.21.15.191:0/614793777 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062824266 0 0) 0x562b6b336000 con 0x562b6a568c00 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28795b700 1 -- 172.21.15.191:0/614793777 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x562b6a893800 con 0x562b6a568c00 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28895d700 1 -- 172.21.15.191:0/614793777 >> v1:172.21.15.138:6789/0 conn(0x562b6a568c00 legacy=0x562b6a654800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28895d700 1 -- 172.21.15.191:0/614793777 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a591d40 con 0x562b6a568800 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28895d700 1 --2- 172.21.15.191:0/614793777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x562b6a84b9e0 tx=0x562b6a659980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd73d7478ae2954 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28795b700 1 -- 172.21.15.191:0/614793777 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a6214a0 con 0x562b6a568800 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28795b700 1 -- 172.21.15.191:0/614793777 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a893500 con 0x562b6a568800 2024-04-27T04:34:44.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.648+0000 7fa28795b700 1 -- 172.21.15.191:0/614793777 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a621600 con 0x562b6a568800 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/614793777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 msgr2=0x562b6a850580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/614793777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x562b6a84b9e0 tx=0x562b6a659980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/614793777 shutdown_connections 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/614793777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/614793777 >> 172.21.15.191:0/614793777 conn(0x562b6a568400 msgr2=0x562b6a654000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/614793777 shutdown_connections 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.649+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/614793777 wait complete. 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28a9b3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28a9b3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a60a500 con 0x562b6a568800 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28895d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28895d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39172/0 (socket says 172.21.15.191:39172) 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28895d700 1 -- 172.21.15.191:0/3333284413 learned_addr learned my addr 172.21.15.191:0/3333284413 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28895d700 1 -- 172.21.15.191:0/3333284413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a591ba0 con 0x562b6a568800 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28895d700 1 --2- 172.21.15.191:0/3333284413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x562b6b339800 tx=0x562b6a659800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e1d8c81938b49d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28715a700 1 -- 172.21.15.191:0/3333284413 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a621760 con 0x562b6a568800 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28715a700 1 -- 172.21.15.191:0/3333284413 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a893980 con 0x562b6a568800 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.650+0000 7fa28715a700 1 -- 172.21.15.191:0/3333284413 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a6214a0 con 0x562b6a568800 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/3333284413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 msgr2=0x562b6a850580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/3333284413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x562b6b339800 tx=0x562b6a659800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/3333284413 shutdown_connections 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/3333284413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/3333284413 >> 172.21.15.191:0/3333284413 conn(0x562b6a568400 msgr2=0x562b6a655000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/3333284413 shutdown_connections 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/3333284413 wait complete. 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28a9b3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a60a3c0 con 0x562b6a568800 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28895d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28895d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39188/0 (socket says 172.21.15.191:39188) 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.651+0000 7fa28895d700 1 -- 172.21.15.191:0/2832348975 learned_addr learned my addr 172.21.15.191:0/2832348975 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa28895d700 1 -- 172.21.15.191:0/2832348975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a591a00 con 0x562b6a568800 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa28895d700 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x562b6b339ec0 tx=0x562b6a659980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa285957700 1 -- 172.21.15.191:0/2832348975 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a621600 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa285957700 1 -- 172.21.15.191:0/2832348975 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a893b00 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b6a591860 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa285957700 1 -- 172.21.15.191:0/2832348975 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a6218c0 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b6a5916c0 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.652+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa285957700 1 -- 172.21.15.191:0/2832348975 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562b6b30ab00 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28a9b3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28a9b3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a60a000 con 0x562b6a569400 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa285957700 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8000 0x562b6a851080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa285957700 1 -- 172.21.15.191:0/2832348975 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x562b6a5e3180 con 0x562b6a568800 2024-04-27T04:34:44.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28915e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28915e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39202/0 (socket says 172.21.15.191:39202) 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28915e700 1 -- 172.21.15.191:0/1195785778 learned_addr learned my addr 172.21.15.191:0/1195785778 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28895d700 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8000 0x562b6a851080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28915e700 1 -- 172.21.15.191:0/1195785778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a590000 con 0x562b6a569400 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28895d700 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8000 0x562b6a851080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x562b6b3fcf90 tx=0x562b6a8bec60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa28915e700 1 --2- 172.21.15.191:0/1195785778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x562b6b3fc210 tx=0x562b6a8beb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10cd9d8a1347b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa283152700 1 -- 172.21.15.191:0/1195785778 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a621600 con 0x562b6a569400 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa283152700 1 -- 172.21.15.191:0/1195785778 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a609c80 con 0x562b6a569400 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.653+0000 7fa283152700 1 -- 172.21.15.191:0/1195785778 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a6218c0 con 0x562b6a569400 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1195785778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 msgr2=0x562b6a850b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/1195785778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x562b6b3fc210 tx=0x562b6a8beb40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1195785778 shutdown_connections 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/1195785778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1195785778 >> 172.21.15.191:0/1195785778 conn(0x562b6a568c00 msgr2=0x562b6a655800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1195785778 shutdown_connections 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1195785778 wait complete. 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28a9b3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a5a4640 con 0x562b6a569400 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28915e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28915e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39216/0 (socket says 172.21.15.191:39216) 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.654+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 learned_addr learned my addr 172.21.15.191:0/2805307619 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a590ea0 con 0x562b6a569400 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28915e700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x562b6b3fc7e0 tx=0x562b6a8bea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28194f700 1 -- 172.21.15.191:0/2805307619 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a621760 con 0x562b6a569400 2024-04-27T04:34:44.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28194f700 1 -- 172.21.15.191:0/2805307619 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a609b00 con 0x562b6a569400 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b6a590820 con 0x562b6a569400 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28194f700 1 -- 172.21.15.191:0/2805307619 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a621600 con 0x562b6a569400 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.655+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b6a5909c0 con 0x562b6a569400 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28194f700 1 -- 172.21.15.191:0/2805307619 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562b6b30adc0 con 0x562b6a569400 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28194f700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8c00 0x562b6a851600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28194f700 1 -- 172.21.15.191:0/2805307619 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x562b6a5e3340 con 0x562b6a569400 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28815c700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8c00 0x562b6a851600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562b6a569800 0x562b6a851b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a569c00 con 0x562b6a569800 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28915e700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562b6a569800 0x562b6a851b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28815c700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8c00 0x562b6a851600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562b6b338030 tx=0x562b6a9a40c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.656+0000 7fa28915e700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562b6a569800 0x562b6a851b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 realm 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9000 con 0x562b6a569800 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9400 con 0x562b6a569800 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9800 con 0x562b6a569800 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562b6a8f9c00 0x562b6a852100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.657+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9000 con 0x562b6a8f9c00 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28895d700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562b6a8f9c00 0x562b6a852100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28895d700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562b6a8f9c00 0x562b6a852100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28a9b3a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28a9b3a80 20 searching for the correct realm 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9400 con 0x562b6a8f9c00 2024-04-27T04:34:44.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28895d700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562b6a8f9400 0x562b6a852c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.658+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9000 con 0x562b6a8f9400 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.659+0000 7fa28815c700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562b6a8f9400 0x562b6a852c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.659+0000 7fa28815c700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562b6a8f9400 0x562b6a852c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.659+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.659+0000 7fa28815c700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562b6a8f9000 0x562b6a853180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.659+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.659+0000 7fa28915e700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562b6a8f9000 0x562b6a853180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562b6a8f9000 0x562b6a853180 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.660+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9400 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.661+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9400 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.662+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.663+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9400 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.664+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.665+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9400 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9c00 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.666+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9400 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a569c00 con 0x562b6a569800 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba400 con 0x562b6a569800 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba800 con 0x562b6a8f9c00 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 zone default found 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 4 Realm: () 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 zonegroup default 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.667+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bac00 con 0x562b6a8f9000 2024-04-27T04:34:44.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb000 con 0x562b6a8f9c00 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb400 con 0x562b6a8f9c00 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 started sync module instance, tier type = 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.668+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb800 con 0x562b6a8f9400 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.669+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.669+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94744470649600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba800 con 0x562b6a8f9400 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.669+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb000 con 0x562b6a569800 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.670+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94744470649600] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.670+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.670+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94744470650752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb400 con 0x562b6a569800 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.670+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba400 con 0x562b6a8f9c00 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.671+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94744470650752] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.671+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.671+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94744470651904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bac00 con 0x562b6a8f9c00 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.671+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a569c00 con 0x562b6a8f9400 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.672+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94744470651904] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.672+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.672+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94744470653056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a8f9800 con 0x562b6a8f9400 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.672+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a569800 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.673+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94744470653056] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.673+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.674+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94744470654208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb800 con 0x562b6a569800 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.674+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba800 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.674+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.674+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94744470655360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bbc00 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.674+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb000 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.675+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94744470654208] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.675+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.675+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94744470656512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb400 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.675+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba000 con 0x562b6a8f9000 2024-04-27T04:34:44.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.675+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94744470655360] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94744470656512] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94744470657664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb800 con 0x562b6a8f9000 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=0 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=1 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=2 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=3 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=4 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=5 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.676+0000 7fa28a9b3a80 20 add_watcher() i=6 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.677+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94744470657664] v31'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.677+0000 7fa28a9b3a80 20 add_watcher() i=7 2024-04-27T04:34:44.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.677+0000 7fa28a9b3a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.677+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba800 con 0x562b6a569800 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.677+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.677+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94744472215552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bbc00 con 0x562b6a569800 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.678+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94744472215552] v31'80 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.678+0000 7fa26f92b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28a9b3a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28a9b3a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28a9b3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28a9b3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a5a4780 con 0x562b6a9bb400 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28815c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28815c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39218/0 (socket says 172.21.15.191:39218) 2024-04-27T04:34:44.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28815c700 1 -- 172.21.15.191:0/2014914729 learned_addr learned my addr 172.21.15.191:0/2014914729 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28815c700 1 -- 172.21.15.191:0/2014914729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a590b60 con 0x562b6a9bb400 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.679+0000 7fa28815c700 1 --2- 172.21.15.191:0/2014914729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x562b6a95f380 tx=0x562b6a8bff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0fcafad170d4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa26f12a700 1 -- 172.21.15.191:0/2014914729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a9d6000 con 0x562b6a9bb400 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa26f12a700 1 -- 172.21.15.191:0/2014914729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a9ce180 con 0x562b6a9bb400 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa26f12a700 1 -- 172.21.15.191:0/2014914729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a9d6160 con 0x562b6a9bb400 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2014914729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 msgr2=0x562b6a852680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2014914729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x562b6a95f380 tx=0x562b6a8bff80 comp rx=0 tx=0).stop 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2014914729 shutdown_connections 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2014914729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2014914729 >> 172.21.15.191:0/2014914729 conn(0x562b6a9bb000 msgr2=0x562b6a656000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2014914729 shutdown_connections 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.680+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2014914729 wait complete. 2024-04-27T04:34:44.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28a9b3a80 1 Processor -- start 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28a9b3a80 1 -- start start 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28a9b3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28a9b3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a60a000 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28815c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28815c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39234/0 (socket says 172.21.15.191:39234) 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28815c700 1 -- 172.21.15.191:0/1529708019 learned_addr learned my addr 172.21.15.191:0/1529708019 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28815c700 1 -- 172.21.15.191:0/1529708019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a590d00 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa28815c700 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562b6a9d9200 tx=0x562b6a9a44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.681+0000 7fa26d927700 1 -- 172.21.15.191:0/1529708019 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a9d62c0 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa26d927700 1 -- 172.21.15.191:0/1529708019 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b6a9ce300 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b6a590340 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa26d927700 1 -- 172.21.15.191:0/1529708019 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b6a9d6000 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b6a590b60 con 0x562b6a9bb400 2024-04-27T04:34:44.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa26d927700 1 -- 172.21.15.191:0/1529708019 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562b6b30bb80 con 0x562b6a9bb400 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa26d927700 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6aa5c000 0x562b6a853700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa26d927700 1 -- 172.21.15.191:0/1529708019 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x562b6a5e3500 con 0x562b6a9bb400 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa28915e700 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6aa5c000 0x562b6a853700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.682+0000 7fa28a9b3a80 5 note: GC not initialized 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.683+0000 7fa28915e700 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6aa5c000 0x562b6a853700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562b6a95fda0 tx=0x562b6a9a4360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.683+0000 7fa26b122700 20 reqs_thread_entry: start 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.683+0000 7fa28a9b3a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa28a9b3a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa26911e700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:14 2024 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa2688fc700 20 reqs_thread_entry: start 2024-04-27T04:34:44.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa28a9b3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9bb800 con 0x562b6a569800 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.684+0000 7fa28a9b3a80 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.703 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.704 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo1", 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.705 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.706 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.707 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.685+0000 7fa26f92b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.685+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94744472215552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6a9ba800 con 0x562b6a569800 2024-04-27T04:34:44.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.686+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94744472215552] v31'81 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.686+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94744470649600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5c400 con 0x562b6a8f9400 2024-04-27T04:34:44.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.687+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94744470649600] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.687+0000 7fa28a9b3a80 20 remove_watcher() i=0 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.687+0000 7fa28a9b3a80 2 removed watcher, disabling cache 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.687+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94744470650752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5c800 con 0x562b6a569800 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.688+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94744470650752] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.688+0000 7fa28a9b3a80 20 remove_watcher() i=1 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.688+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94744470651904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5cc00 con 0x562b6a8f9c00 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.689+0000 7fa28895d700 1 -- 172.21.15.191:0/2805307619 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94744470651904] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3200 con 0x562b6a8f9c00 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.689+0000 7fa28a9b3a80 20 remove_watcher() i=2 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.689+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94744470653056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5d000 con 0x562b6a8f9400 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.690+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94744470653056] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b3440 con 0x562b6a8f9400 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.690+0000 7fa28a9b3a80 20 remove_watcher() i=3 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.690+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94744470654208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5d400 con 0x562b6a569800 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.691+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94744470654208] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a569800 2024-04-27T04:34:44.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.691+0000 7fa28a9b3a80 20 remove_watcher() i=4 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.691+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94744470655360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5d800 con 0x562b6a8f9000 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.692+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94744470655360] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.692+0000 7fa28a9b3a80 20 remove_watcher() i=5 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.692+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94744470656512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5dc00 con 0x562b6a8f9000 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.693+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94744470656512] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.693+0000 7fa28a9b3a80 20 remove_watcher() i=6 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.693+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94744470657664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562b6aa5c800 con 0x562b6a8f9000 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28915e700 1 -- 172.21.15.191:0/2805307619 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94744470657664] v31'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b6a9b2900 con 0x562b6a8f9000 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 20 remove_watcher() i=7 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562b6a8f9000 msgr2=0x562b6a853180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562b6a8f9000 0x562b6a853180 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562b6a569800 msgr2=0x562b6a851b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562b6a569800 0x562b6a851b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562b6a8f9400 msgr2=0x562b6a852c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562b6a8f9400 0x562b6a852c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562b6a8f9c00 msgr2=0x562b6a852100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562b6a8f9c00 0x562b6a852100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8c00 msgr2=0x562b6a851600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8c00 0x562b6a851600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562b6b338030 tx=0x562b6a9a40c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 msgr2=0x562b6a850b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2805307619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a569400 0x562b6a850b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x562b6b3fc7e0 tx=0x562b6a8bea80 comp rx=0 tx=0).stop 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 reap_dead start 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 shutdown_connections 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28815c700 1 -- 172.21.15.191:0/2805307619 reap_dead start 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 >> 172.21.15.191:0/2805307619 conn(0x562b6a568c00 msgr2=0x562b6a655800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 shutdown_connections 2024-04-27T04:34:44.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.694+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2805307619 wait complete. 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6aa5c000 msgr2=0x562b6a853700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6aa5c000 0x562b6a853700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562b6a95fda0 tx=0x562b6a9a4360 comp rx=0 tx=0).stop 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 msgr2=0x562b6a852680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562b6a9d9200 tx=0x562b6a9a44e0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 shutdown_connections 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6aa5c000 0x562b6a853700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/1529708019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a9bb400 0x562b6a852680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 >> 172.21.15.191:0/1529708019 conn(0x562b6a9bb000 msgr2=0x562b6a656000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 shutdown_connections 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/1529708019 wait complete. 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8000 msgr2=0x562b6a851080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8000 0x562b6a851080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x562b6b3fcf90 tx=0x562b6a8bec60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 msgr2=0x562b6a850580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x562b6b339ec0 tx=0x562b6a659980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 shutdown_connections 2024-04-27T04:34:44.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562b6a8f8000 0x562b6a851080 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 --2- 172.21.15.191:0/2832348975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b6a568800 0x562b6a850580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 >> 172.21.15.191:0/2832348975 conn(0x562b6a568400 msgr2=0x562b6a655000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 shutdown_connections 2024-04-27T04:34:44.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.695+0000 7fa28a9b3a80 1 -- 172.21.15.191:0/2832348975 wait complete. 2024-04-27T04:34:44.717 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.717 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.718 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:44.764 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.764 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf65485a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf65485a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ca17d2f500 con 0x55ca17a04c00 2024-04-27T04:34:44.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf65485a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ca17aa6a00 con 0x55ca17a04800 2024-04-27T04:34:44.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf6342f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf6342f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39248/0 (socket says 172.21.15.191:39248) 2024-04-27T04:34:44.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf6342f700 1 -- 172.21.15.191:0/1133906349 learned_addr learned my addr 172.21.15.191:0/1133906349 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.763+0000 7fbf62c2e700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55ca17a04c00 0x55ca17af0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55376/0 (socket says 172.21.15.191:55376) 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf6342f700 1 -- 172.21.15.191:0/1133906349 >> v1:172.21.15.138:6789/0 conn(0x55ca17a04c00 legacy=0x55ca17af0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf6342f700 1 -- 172.21.15.191:0/1133906349 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2dd40 con 0x55ca17a04800 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf6342f700 1 --2- 172.21.15.191:0/1133906349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ca17ce79e0 tx=0x55ca17af5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398af9f94017acf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf6242d700 1 -- 172.21.15.191:0/1133906349 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd4a0 con 0x55ca17a04800 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf6242d700 1 -- 172.21.15.191:0/1133906349 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17d2f800 con 0x55ca17a04800 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf6242d700 1 -- 172.21.15.191:0/1133906349 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd600 con 0x55ca17a04800 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 -- 172.21.15.191:0/1133906349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 msgr2=0x55ca17cec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 --2- 172.21.15.191:0/1133906349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ca17ce79e0 tx=0x55ca17af5920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 -- 172.21.15.191:0/1133906349 shutdown_connections 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 --2- 172.21.15.191:0/1133906349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 -- 172.21.15.191:0/1133906349 >> 172.21.15.191:0/1133906349 conn(0x55ca17a04400 msgr2=0x55ca17af0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 -- 172.21.15.191:0/1133906349 shutdown_connections 2024-04-27T04:34:44.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.764+0000 7fbf65485a80 1 -- 172.21.15.191:0/1133906349 wait complete. 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.765+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.765+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.765+0000 7fbf65485a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.765+0000 7fbf65485a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17aa6500 con 0x55ca17a04800 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf6342f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf6342f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39260/0 (socket says 172.21.15.191:39260) 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf6342f700 1 -- 172.21.15.191:0/3000275574 learned_addr learned my addr 172.21.15.191:0/3000275574 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf6342f700 1 -- 172.21.15.191:0/3000275574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2dba0 con 0x55ca17a04800 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf6342f700 1 --2- 172.21.15.191:0/3000275574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55ca17ce6d20 tx=0x55ca17af5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb183e5f5e6273 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf61c2c700 1 -- 172.21.15.191:0/3000275574 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd760 con 0x55ca17a04800 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf61c2c700 1 -- 172.21.15.191:0/3000275574 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17d2fb00 con 0x55ca17a04800 2024-04-27T04:34:44.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf61c2c700 1 -- 172.21.15.191:0/3000275574 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd4a0 con 0x55ca17a04800 2024-04-27T04:34:44.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf65485a80 1 -- 172.21.15.191:0/3000275574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 msgr2=0x55ca17cec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf65485a80 1 --2- 172.21.15.191:0/3000275574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55ca17ce6d20 tx=0x55ca17af5800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf65485a80 1 -- 172.21.15.191:0/3000275574 shutdown_connections 2024-04-27T04:34:44.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf65485a80 1 --2- 172.21.15.191:0/3000275574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.766+0000 7fbf65485a80 1 -- 172.21.15.191:0/3000275574 >> 172.21.15.191:0/3000275574 conn(0x55ca17a04400 msgr2=0x55ca17af1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf65485a80 1 -- 172.21.15.191:0/3000275574 shutdown_connections 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf65485a80 1 -- 172.21.15.191:0/3000275574 wait complete. 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf65485a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf65485a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17aa63c0 con 0x55ca17a04800 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf6342f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf6342f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39274/0 (socket says 172.21.15.191:39274) 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf6342f700 1 -- 172.21.15.191:0/3123514356 learned_addr learned my addr 172.21.15.191:0/3123514356 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf6342f700 1 -- 172.21.15.191:0/3123514356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2da00 con 0x55ca17a04800 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.767+0000 7fbf6342f700 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55ca187cf890 tx=0x55ca17af5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf60429700 1 -- 172.21.15.191:0/3123514356 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd600 con 0x55ca17a04800 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf60429700 1 -- 172.21.15.191:0/3123514356 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17d2fc80 con 0x55ca17a04800 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf60429700 1 -- 172.21.15.191:0/3123514356 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd8c0 con 0x55ca17a04800 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca17a2d860 con 0x55ca17a04800 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca17a2d6c0 con 0x55ca17a04800 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf60429700 1 -- 172.21.15.191:0/3123514356 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca187a6b00 con 0x55ca17a04800 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf65485a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf65485a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17aa6000 con 0x55ca17a05400 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf60429700 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8c000 0x55ca17ced080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.768+0000 7fbf60429700 1 -- 172.21.15.191:0/3123514356 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ca17a7f180 con 0x55ca17a04800 2024-04-27T04:34:44.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf6342f700 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8c000 0x55ca17ced080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf63c30700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf63c30700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39280/0 (socket says 172.21.15.191:39280) 2024-04-27T04:34:44.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf63c30700 1 -- 172.21.15.191:0/1294784976 learned_addr learned my addr 172.21.15.191:0/1294784976 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf63c30700 1 -- 172.21.15.191:0/1294784976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2c000 con 0x55ca17a05400 2024-04-27T04:34:44.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf6342f700 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8c000 0x55ca17ced080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55ca188909f0 tx=0x55ca17d52b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf63c30700 1 --2- 172.21.15.191:0/1294784976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ca188905a0 tx=0x55ca17d52e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeaf4dab539aa5b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf5dc24700 1 -- 172.21.15.191:0/1294784976 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd600 con 0x55ca17a05400 2024-04-27T04:34:44.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf5dc24700 1 -- 172.21.15.191:0/1294784976 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17aa5c80 con 0x55ca17a05400 2024-04-27T04:34:44.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf5dc24700 1 -- 172.21.15.191:0/1294784976 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd8c0 con 0x55ca17a05400 2024-04-27T04:34:44.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf65485a80 1 -- 172.21.15.191:0/1294784976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 msgr2=0x55ca17cecb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf65485a80 1 --2- 172.21.15.191:0/1294784976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ca188905a0 tx=0x55ca17d52e40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf65485a80 1 -- 172.21.15.191:0/1294784976 shutdown_connections 2024-04-27T04:34:44.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf65485a80 1 --2- 172.21.15.191:0/1294784976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.769+0000 7fbf65485a80 1 -- 172.21.15.191:0/1294784976 >> 172.21.15.191:0/1294784976 conn(0x55ca17a04c00 msgr2=0x55ca17af1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf65485a80 1 -- 172.21.15.191:0/1294784976 shutdown_connections 2024-04-27T04:34:44.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf65485a80 1 -- 172.21.15.191:0/1294784976 wait complete. 2024-04-27T04:34:44.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf65485a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf65485a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17a40640 con 0x55ca17a05400 2024-04-27T04:34:44.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf63c30700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf63c30700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39296/0 (socket says 172.21.15.191:39296) 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.770+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 learned_addr learned my addr 172.21.15.191:0/971860306 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2cea0 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf63c30700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ca18891e90 tx=0x55ca17d52a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf5c421700 1 -- 172.21.15.191:0/971860306 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd760 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf5c421700 1 -- 172.21.15.191:0/971860306 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17aa5b00 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca17a2c820 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf5c421700 1 -- 172.21.15.191:0/971860306 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17abd600 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca17a2c9c0 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.771+0000 7fbf5c421700 1 -- 172.21.15.191:0/971860306 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca187a6dc0 con 0x55ca17a05400 2024-04-27T04:34:44.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf5c421700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8cc00 0x55ca17ced600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf5c421700 1 -- 172.21.15.191:0/971860306 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ca17a7f340 con 0x55ca17a05400 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8cc00 0x55ca17ced600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8cc00 0x55ca17ced600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55ca17ce7f80 tx=0x55ca17e3a0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ca17a05800 0x55ca17cedb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a05c00 con 0x55ca17a05800 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.772+0000 7fbf63c30700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ca17a05800 0x55ca17cedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf63c30700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ca17a05800 0x55ca17cedb80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 realm 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d000 con 0x55ca17a05800 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d400 con 0x55ca17a05800 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d800 con 0x55ca17a05800 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.773+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ca17d8dc00 0x55ca17cee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d000 con 0x55ca17d8dc00 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf6342f700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ca17d8dc00 0x55ca17cee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf6342f700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ca17d8dc00 0x55ca17cee100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 20 searching for the correct realm 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.774+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d400 con 0x55ca17d8dc00 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.775+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.775+0000 7fbf6342f700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ca17d8d400 0x55ca17ceec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.775+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d000 con 0x55ca17d8d400 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.775+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ca17d8d400 0x55ca17ceec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.775+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ca17d8d400 0x55ca17ceec00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.776+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.776+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ca17d8d000 0x55ca17cef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.776+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.776+0000 7fbf63c30700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ca17d8d000 0x55ca17cef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.776+0000 7fbf63c30700 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ca17d8d000 0x55ca17cef180 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.777+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d400 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.778+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d400 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.779+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.780+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d400 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.781+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d400 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8dc00 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.782+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d400 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a05c00 con 0x55ca17a05800 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.783+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56400 con 0x55ca17a05800 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56800 con 0x55ca17d8dc00 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 zone default found 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 4 Realm: () 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 zonegroup default 2024-04-27T04:34:44.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56c00 con 0x55ca17d8d000 2024-04-27T04:34:44.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57000 con 0x55ca17d8dc00 2024-04-27T04:34:44.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.784+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57400 con 0x55ca17d8dc00 2024-04-27T04:34:44.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.785+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.785+0000 7fbf65485a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.785+0000 7fbf65485a80 20 started sync module instance, tier type = 2024-04-27T04:34:44.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.785+0000 7fbf65485a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.785+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57800 con 0x55ca17d8d400 2024-04-27T04:34:44.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.786+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.786+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94326471146240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56800 con 0x55ca17d8d400 2024-04-27T04:34:44.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.786+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57000 con 0x55ca17a05800 2024-04-27T04:34:44.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.787+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94326471146240] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.787+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.787+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94326471147392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57400 con 0x55ca17a05800 2024-04-27T04:34:44.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.787+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56400 con 0x55ca17d8dc00 2024-04-27T04:34:44.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.788+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94326471147392] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.788+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.788+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94326471148544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56c00 con 0x55ca17d8dc00 2024-04-27T04:34:44.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.788+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a05c00 con 0x55ca17d8d400 2024-04-27T04:34:44.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.789+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.789+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94326471149696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17d8d800 con 0x55ca17d8d400 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.789+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17a05800 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.789+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94326471148544] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.790+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.790+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94326471150848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57800 con 0x55ca17a05800 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.790+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56800 con 0x55ca17d8d000 2024-04-27T04:34:44.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.790+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94326471149696] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.791+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.791+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94326471152000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57c00 con 0x55ca17d8d000 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.791+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57000 con 0x55ca17d8d000 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.791+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94326471150848] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.792+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.792+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94326471152000] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.792+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94326471153152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57400 con 0x55ca17d8d000 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.792+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56000 con 0x55ca17d8d000 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94326471153152] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94326471154304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57800 con 0x55ca17d8d000 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=0 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=1 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=2 2024-04-27T04:34:44.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=3 2024-04-27T04:34:44.834 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=4 2024-04-27T04:34:44.834 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=5 2024-04-27T04:34:44.834 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.793+0000 7fbf65485a80 20 add_watcher() i=6 2024-04-27T04:34:44.834 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.794+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94326471154304] v31'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.834 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.794+0000 7fbf65485a80 20 add_watcher() i=7 2024-04-27T04:34:44.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.794+0000 7fbf65485a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.794+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56800 con 0x55ca17a05800 2024-04-27T04:34:44.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.794+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.794+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94326472712192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57c00 con 0x55ca17a05800 2024-04-27T04:34:44.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94326472712192] v31'82 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf4a3fd700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf65485a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf65485a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf65485a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.795+0000 7fbf65485a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17a40780 con 0x55ca17e57400 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf62c2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf62c2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39312/0 (socket says 172.21.15.191:39312) 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf62c2e700 1 -- 172.21.15.191:0/809338530 learned_addr learned my addr 172.21.15.191:0/809338530 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf62c2e700 1 -- 172.21.15.191:0/809338530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2cb60 con 0x55ca17e57400 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/809338530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ca17df57a0 tx=0x55ca17d53f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdde4923a1b1605b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf49bfc700 1 -- 172.21.15.191:0/809338530 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17e76000 con 0x55ca17e57400 2024-04-27T04:34:44.837 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf49bfc700 1 -- 172.21.15.191:0/809338530 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17e78000 con 0x55ca17e57400 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.796+0000 7fbf49bfc700 1 -- 172.21.15.191:0/809338530 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17e76160 con 0x55ca17e57400 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 -- 172.21.15.191:0/809338530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 msgr2=0x55ca17cee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 --2- 172.21.15.191:0/809338530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ca17df57a0 tx=0x55ca17d53f80 comp rx=0 tx=0).stop 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 -- 172.21.15.191:0/809338530 shutdown_connections 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 --2- 172.21.15.191:0/809338530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 -- 172.21.15.191:0/809338530 >> 172.21.15.191:0/809338530 conn(0x55ca17e57000 msgr2=0x55ca17af2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 -- 172.21.15.191:0/809338530 shutdown_connections 2024-04-27T04:34:44.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 -- 172.21.15.191:0/809338530 wait complete. 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 Processor -- start 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.797+0000 7fbf65485a80 1 -- start start 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf65485a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf65485a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17aa6000 con 0x55ca17e57400 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf62c2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf62c2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39314/0 (socket says 172.21.15.191:39314) 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf62c2e700 1 -- 172.21.15.191:0/363893994 learned_addr learned my addr 172.21.15.191:0/363893994 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf62c2e700 1 -- 172.21.15.191:0/363893994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca17a2cd00 con 0x55ca17e57400 2024-04-27T04:34:44.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf62c2e700 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55ca17e6e1e0 tx=0x55ca17e3a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf483f9700 1 -- 172.21.15.191:0/363893994 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17e762c0 con 0x55ca17e57400 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf483f9700 1 -- 172.21.15.191:0/363893994 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca17e78180 con 0x55ca17e57400 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca17a2c340 con 0x55ca17e57400 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf483f9700 1 -- 172.21.15.191:0/363893994 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca17e76000 con 0x55ca17e57400 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.798+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca17a2cb60 con 0x55ca17e57400 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.799+0000 7fbf483f9700 1 -- 172.21.15.191:0/363893994 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca187a7b80 con 0x55ca17e57400 2024-04-27T04:34:44.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.799+0000 7fbf483f9700 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17ef8000 0x55ca17cef700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.799+0000 7fbf483f9700 1 -- 172.21.15.191:0/363893994 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ca17a7f500 con 0x55ca17e57400 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.799+0000 7fbf63c30700 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17ef8000 0x55ca17cef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.799+0000 7fbf65485a80 5 note: GC not initialized 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.799+0000 7fbf63c30700 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17ef8000 0x55ca17cef700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ca17df5410 tx=0x55ca17e3a360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf45bf4700 20 reqs_thread_entry: start 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf65485a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf65485a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf43bf0700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:44.842 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf433ce700 20 reqs_thread_entry: start 2024-04-27T04:34:44.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf65485a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.800+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e57800 con 0x55ca17a05800 2024-04-27T04:34:44.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.801+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.801+0000 7fbf65485a80 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T04:34:44.843 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo1", 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.801+0000 7fbf4a3fd700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.801+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94326472712192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17e56800 con 0x55ca17a05800 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.802+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94326472712192] v31'83 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.802+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94326471146240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef8400 con 0x55ca17d8d400 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.803+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94326471146240] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.803+0000 7fbf65485a80 20 remove_watcher() i=0 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.803+0000 7fbf65485a80 2 removed watcher, disabling cache 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.803+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94326471147392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef8800 con 0x55ca17a05800 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.804+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94326471147392] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.804+0000 7fbf65485a80 20 remove_watcher() i=1 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.804+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94326471148544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef8c00 con 0x55ca17d8dc00 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.805+0000 7fbf6342f700 1 -- 172.21.15.191:0/971860306 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94326471148544] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4efc0 con 0x55ca17d8dc00 2024-04-27T04:34:44.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.805+0000 7fbf65485a80 20 remove_watcher() i=2 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.805+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94326471149696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef9000 con 0x55ca17d8d400 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.806+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94326471149696] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4f200 con 0x55ca17d8d400 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.806+0000 7fbf65485a80 20 remove_watcher() i=3 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.806+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94326471150848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef9400 con 0x55ca17a05800 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.807+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94326471150848] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17a05800 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.807+0000 7fbf65485a80 20 remove_watcher() i=4 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.807+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94326471152000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef9800 con 0x55ca17d8d000 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.808+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94326471152000] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.808+0000 7fbf65485a80 20 remove_watcher() i=5 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.808+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94326471153152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef9c00 con 0x55ca17d8d000 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.809+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94326471153152] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.809+0000 7fbf65485a80 20 remove_watcher() i=6 2024-04-27T04:34:44.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.809+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94326471154304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17ef8800 con 0x55ca17d8d000 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf63c30700 1 -- 172.21.15.191:0/971860306 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94326471154304] v31'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca17e4e900 con 0x55ca17d8d000 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 20 remove_watcher() i=7 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ca17d8d000 msgr2=0x55ca17cef180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ca17d8d000 0x55ca17cef180 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ca17a05800 msgr2=0x55ca17cedb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ca17a05800 0x55ca17cedb80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ca17d8d400 msgr2=0x55ca17ceec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ca17d8d400 0x55ca17ceec00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ca17d8dc00 msgr2=0x55ca17cee100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ca17d8dc00 0x55ca17cee100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8cc00 msgr2=0x55ca17ced600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8cc00 0x55ca17ced600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55ca17ce7f80 tx=0x55ca17e3a0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 msgr2=0x55ca17cecb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 --2- 172.21.15.191:0/971860306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a05400 0x55ca17cecb00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ca18891e90 tx=0x55ca17d52a20 comp rx=0 tx=0).stop 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 reap_dead start 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 shutdown_connections 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf62c2e700 1 -- 172.21.15.191:0/971860306 reap_dead start 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 >> 172.21.15.191:0/971860306 conn(0x55ca17a04c00 msgr2=0x55ca17af1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 shutdown_connections 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.810+0000 7fbf65485a80 1 -- 172.21.15.191:0/971860306 wait complete. 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17ef8000 msgr2=0x55ca17cef700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17ef8000 0x55ca17cef700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ca17df5410 tx=0x55ca17e3a360 comp rx=0 tx=0).stop 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 msgr2=0x55ca17cee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55ca17e6e1e0 tx=0x55ca17e3a4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 shutdown_connections 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17ef8000 0x55ca17cef700 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/363893994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17e57400 0x55ca17cee680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 >> 172.21.15.191:0/363893994 conn(0x55ca17e57000 msgr2=0x55ca17af2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 shutdown_connections 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/363893994 wait complete. 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8c000 msgr2=0x55ca17ced080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8c000 0x55ca17ced080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55ca188909f0 tx=0x55ca17d52b40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 msgr2=0x55ca17cec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55ca187cf890 tx=0x55ca17af5920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 shutdown_connections 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ca17d8c000 0x55ca17ced080 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 --2- 172.21.15.191:0/3123514356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca17a04800 0x55ca17cec580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 >> 172.21.15.191:0/3123514356 conn(0x55ca17a04400 msgr2=0x55ca17af1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 shutdown_connections 2024-04-27T04:34:44.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.811+0000 7fbf65485a80 1 -- 172.21.15.191:0/3123514356 wait complete. 2024-04-27T04:34:44.858 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.858 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-27T04:34:44.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-27T04:34:44.858 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-04-27T04:34:44.909 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:44.909 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:44.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b752ca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55e8bb830c00 0x55e8bbb18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b752ca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e8bbb5b500 con 0x55e8bb830800 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b752ca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e8bb8d2a00 con 0x55e8bb830c00 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b4cd5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55e8bb830c00 0x55e8bbb18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b54d6700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55e8bb830800 0x55e8bb91c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55388/0 (socket says 172.21.15.191:55388) 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.907+0000 7fd6b54d6700 1 -- 172.21.15.191:0/930181289 learned_addr learned my addr 172.21.15.191:0/930181289 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.908+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/930181289 >> v1:172.21.15.138:6789/0 conn(0x55e8bb830800 legacy=0x55e8bb91c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.908+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/930181289 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb859d40 con 0x55e8bb830c00 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.908+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/930181289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830c00 0x55e8bbb18580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55e8bbb13890 tx=0x55e8bb921920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a75480caa1078 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.908+0000 7fd6b44d4700 1 -- 172.21.15.191:0/930181289 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e94a0 con 0x55e8bb830c00 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.908+0000 7fd6b44d4700 1 -- 172.21.15.191:0/930181289 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bbb5b800 con 0x55e8bb830c00 2024-04-27T04:34:44.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.908+0000 7fd6b44d4700 1 -- 172.21.15.191:0/930181289 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e9600 con 0x55e8bb830c00 2024-04-27T04:34:44.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- 172.21.15.191:0/930181289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830c00 msgr2=0x55e8bbb18580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/930181289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830c00 0x55e8bbb18580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55e8bbb13890 tx=0x55e8bb921920 comp rx=0 tx=0).stop 2024-04-27T04:34:44.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- 172.21.15.191:0/930181289 shutdown_connections 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/930181289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830c00 0x55e8bbb18580 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- 172.21.15.191:0/930181289 >> 172.21.15.191:0/930181289 conn(0x55e8bb830400 msgr2=0x55e8bb91c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- 172.21.15.191:0/930181289 shutdown_connections 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- 172.21.15.191:0/930181289 wait complete. 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b752ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bb8d2500 con 0x55e8bb830800 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.909+0000 7fd6b54d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b54d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39338/0 (socket says 172.21.15.191:39338) 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b54d6700 1 -- 172.21.15.191:0/136205600 learned_addr learned my addr 172.21.15.191:0/136205600 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b54d6700 1 -- 172.21.15.191:0/136205600 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb859ba0 con 0x55e8bb830800 2024-04-27T04:34:44.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/136205600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55e8bbb12a50 tx=0x55e8bb921b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a9b6adc38a3446 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b3cd3700 1 -- 172.21.15.191:0/136205600 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e94a0 con 0x55e8bb830800 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b3cd3700 1 -- 172.21.15.191:0/136205600 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bbb5bb00 con 0x55e8bb830800 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b3cd3700 1 -- 172.21.15.191:0/136205600 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e9600 con 0x55e8bb830800 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 -- 172.21.15.191:0/136205600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 msgr2=0x55e8bbb18580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/136205600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55e8bbb12a50 tx=0x55e8bb921b60 comp rx=0 tx=0).stop 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 -- 172.21.15.191:0/136205600 shutdown_connections 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/136205600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 -- 172.21.15.191:0/136205600 >> 172.21.15.191:0/136205600 conn(0x55e8bb830400 msgr2=0x55e8bb91d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 -- 172.21.15.191:0/136205600 shutdown_connections 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.910+0000 7fd6b752ca80 1 -- 172.21.15.191:0/136205600 wait complete. 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b752ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b752ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bb8d23c0 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b54d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b54d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39352/0 (socket says 172.21.15.191:39352) 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b54d6700 1 -- 172.21.15.191:0/2223641004 learned_addr learned my addr 172.21.15.191:0/2223641004 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b54d6700 1 -- 172.21.15.191:0/2223641004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb859a00 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55e8bc5f7f80 tx=0x55e8bb921a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b24d0700 1 -- 172.21.15.191:0/2223641004 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e98c0 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b24d0700 1 -- 172.21.15.191:0/2223641004 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bb8d1c80 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.911+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e8bb859860 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b24d0700 1 -- 172.21.15.191:0/2223641004 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e94a0 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8bb8596c0 con 0x55e8bb830800 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b24d0700 1 -- 172.21.15.191:0/2223641004 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e8bc5d2b00 con 0x55e8bb830800 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b752ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b752ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bb86c640 con 0x55e8bb831400 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b24d0700 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2000 0x55e8bbb19080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.912+0000 7fd6b24d0700 1 -- 172.21.15.191:0/2223641004 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e8bb8ab180 con 0x55e8bb830800 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b5cd7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b5cd7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39358/0 (socket says 172.21.15.191:39358) 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/14369258 learned_addr learned my addr 172.21.15.191:0/14369258 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2000 0x55e8bbb19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/14369258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb858000 con 0x55e8bb831400 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/14369258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55e8bc6ae930 tx=0x55e8bbb7cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6074ff0fc2fc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2000 0x55e8bbb19080 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55e8bc6af590 tx=0x55e8bbb7c240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6afccb700 1 -- 172.21.15.191:0/14369258 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e98c0 con 0x55e8bb831400 2024-04-27T04:34:44.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6afccb700 1 -- 172.21.15.191:0/14369258 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bbb5a300 con 0x55e8bb831400 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6afccb700 1 -- 172.21.15.191:0/14369258 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e94a0 con 0x55e8bb831400 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 -- 172.21.15.191:0/14369258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 msgr2=0x55e8bbb18b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/14369258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55e8bc6ae930 tx=0x55e8bbb7cde0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 -- 172.21.15.191:0/14369258 shutdown_connections 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/14369258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 -- 172.21.15.191:0/14369258 >> 172.21.15.191:0/14369258 conn(0x55e8bb830c00 msgr2=0x55e8bb91d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 -- 172.21.15.191:0/14369258 shutdown_connections 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.913+0000 7fd6b752ca80 1 -- 172.21.15.191:0/14369258 wait complete. 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b752ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b752ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bb86c8c0 con 0x55e8bb831400 2024-04-27T04:34:44.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b5cd7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b5cd7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39368/0 (socket says 172.21.15.191:39368) 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 learned_addr learned my addr 172.21.15.191:0/568525558 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb858ea0 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55e8bc6ae240 tx=0x55e8bbb7cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6ae4c8700 1 -- 172.21.15.191:0/568525558 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e9b80 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.914+0000 7fd6ae4c8700 1 -- 172.21.15.191:0/568525558 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bb8d1c80 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6ae4c8700 1 -- 172.21.15.191:0/568525558 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e98c0 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e8bb858820 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8bb8589c0 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6ae4c8700 1 -- 172.21.15.191:0/568525558 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e8bc5d2dc0 con 0x55e8bb831400 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6ae4c8700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2c00 0x55e8bbb19600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6ae4c8700 1 -- 172.21.15.191:0/568525558 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e8bb8ab340 con 0x55e8bb831400 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.915+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2c00 0x55e8bbb19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e8bb831800 0x55e8bbb19b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bb831c00 con 0x55e8bb831800 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e8bb831800 0x55e8bbb19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2c00 0x55e8bbb19600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55e8bbb13890 tx=0x55e8bb921800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e8bb831800 0x55e8bbb19b80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 20 realm 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.916+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3000 con 0x55e8bb831800 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3400 con 0x55e8bb831800 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3800 con 0x55e8bb831800 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e8bbbc3c00 0x55e8bbb1a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e8bbbc3c00 0x55e8bbb1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.917+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e8bbbc3c00 0x55e8bbb1a100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b752ca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b752ca80 20 searching for the correct realm 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3400 con 0x55e8bbbc3c00 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b54d6700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e8bbbc3400 0x55e8bbb1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3000 con 0x55e8bbbc3400 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e8bbbc3400 0x55e8bbb1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.918+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e8bbbc3400 0x55e8bbb1ac00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.919+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.919+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e8bbbc3000 0x55e8bbb1b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.919+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.919+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e8bbbc3000 0x55e8bbb1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.919+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e8bbbc3000 0x55e8bbb1b180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.919+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.920+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3400 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.921+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3400 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.922+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.923+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3400 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.924+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3400 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.925+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3400 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:44.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bb831c00 con 0x55e8bb831800 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86400 con 0x55e8bb831800 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.926+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86800 con 0x55e8bbbc3c00 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 zone default found 2024-04-27T04:34:44.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 4 Realm: () 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 zonegroup default 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86c00 con 0x55e8bbbc3000 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87000 con 0x55e8bbbc3c00 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87400 con 0x55e8bbbc3c00 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 started sync module instance, tier type = 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.927+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87800 con 0x55e8bbbc3400 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.928+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.928+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94458069711616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86800 con 0x55e8bbbc3400 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.928+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87000 con 0x55e8bb831800 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.929+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94458069711616] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.930+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.930+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94458069712768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87400 con 0x55e8bb831800 2024-04-27T04:34:44.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.930+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86400 con 0x55e8bbbc3c00 2024-04-27T04:34:44.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.930+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94458069712768] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.931+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.931+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94458069713920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86c00 con 0x55e8bbbc3c00 2024-04-27T04:34:44.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.931+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bb831c00 con 0x55e8bbbc3400 2024-04-27T04:34:44.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94458069713920] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94458069715072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbbc3800 con 0x55e8bbbc3400 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bb831800 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94458069716224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87800 con 0x55e8bb831800 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.932+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86800 con 0x55e8bbbc3000 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.933+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94458069715072] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.933+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94458069716224] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.933+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.933+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94458069717376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87c00 con 0x55e8bbbc3000 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.933+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87000 con 0x55e8bbbc3000 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.934+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.934+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94458069718528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87400 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.934+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86000 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.934+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94458069717376] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94458069718528] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94458069719680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87800 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=0 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=1 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=2 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=3 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=4 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=5 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.935+0000 7fd6b752ca80 20 add_watcher() i=6 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.936+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94458069719680] v31'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.936+0000 7fd6b752ca80 20 add_watcher() i=7 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.936+0000 7fd6b752ca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.936+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86800 con 0x55e8bb831800 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.936+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.936+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94458071302144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87c00 con 0x55e8bb831800 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.937+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94458071302144] v31'84 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.937+0000 7fd69c4a4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.937+0000 7fd6b752ca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.937+0000 7fd6b752ca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b752ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b752ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bb8d23c0 con 0x55e8bbc87400 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b4cd5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b4cd5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39384/0 (socket says 172.21.15.191:39384) 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/3973331244 learned_addr learned my addr 172.21.15.191:0/3973331244 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/3973331244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb858b60 con 0x55e8bbc87400 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.938+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/3973331244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55e8bbc9c060 tx=0x55e8bb921980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a8fd31846b396f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd69bca3700 1 -- 172.21.15.191:0/3973331244 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bb8e9760 con 0x55e8bbc87400 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd69bca3700 1 -- 172.21.15.191:0/3973331244 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bbca8000 con 0x55e8bbc87400 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd69bca3700 1 -- 172.21.15.191:0/3973331244 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bbcaa000 con 0x55e8bbc87400 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3973331244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 msgr2=0x55e8bbb1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/3973331244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55e8bbc9c060 tx=0x55e8bb921980 comp rx=0 tx=0).stop 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3973331244 shutdown_connections 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/3973331244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3973331244 >> 172.21.15.191:0/3973331244 conn(0x55e8bbc87000 msgr2=0x55e8bb91e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3973331244 shutdown_connections 2024-04-27T04:34:44.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.939+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3973331244 wait complete. 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b752ca80 1 Processor -- start 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b752ca80 1 -- start start 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b752ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b752ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bb8d2500 con 0x55e8bbc87400 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b4cd5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b4cd5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39386/0 (socket says 172.21.15.191:39386) 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/3266268801 learned_addr learned my addr 172.21.15.191:0/3266268801 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/3266268801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bb858d00 con 0x55e8bbc87400 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b4cd5700 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55e8bbc9c270 tx=0x55e8bb9218c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd69a4a0700 1 -- 172.21.15.191:0/3266268801 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bbcaa160 con 0x55e8bbc87400 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd69a4a0700 1 -- 172.21.15.191:0/3266268801 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8bbca8180 con 0x55e8bbc87400 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e8bb858340 con 0x55e8bbc87400 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd69a4a0700 1 -- 172.21.15.191:0/3266268801 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e8bbcaa2c0 con 0x55e8bbc87400 2024-04-27T04:34:44.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.940+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8bb859d40 con 0x55e8bbc87400 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.941+0000 7fd69a4a0700 1 -- 172.21.15.191:0/3266268801 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e8bc5d3b80 con 0x55e8bbc87400 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.941+0000 7fd69a4a0700 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbd22000 0x55e8bbb1b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.941+0000 7fd69a4a0700 1 -- 172.21.15.191:0/3266268801 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e8bb8ab500 con 0x55e8bbc87400 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.941+0000 7fd6b752ca80 5 note: GC not initialized 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.941+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbd22000 0x55e8bbb1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.942+0000 7fd6b5cd7700 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbd22000 0x55e8bbb1b700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55e8bbc29fb0 tx=0x55e8bbcab320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.942+0000 7fd697c9b700 20 reqs_thread_entry: start 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.942+0000 7fd6b752ca80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd695c97700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:44.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd695475700 20 reqs_thread_entry: start 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc87800 con 0x55e8bb831800 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbc86800 con 0x55e8bbbc3400 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:44.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.943+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~621 in=621b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd22800 con 0x55e8bb831800 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~621] v31'8 uv8 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b752ca80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b752ca80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b752ca80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94458071304448 in=809b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd22c00 con 0x55e8bb831800 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94458069716224 notify 133143986183 ret 0) v3 ==== 839+0+0 (crc 0 0 0) 0x55e8bbc2bd40 con 0x55e8bb831800 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6afccb700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986183 cookie 94458069716224 notifier 4404 bl.length()=797 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6afccb700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6afccb700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(65 notify.4 [notify cookie 94458071304448 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6afccb700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd96000 con 0x55e8bb831800 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94458071304448 notify 133143986183 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e8bbcebba0 con 0x55e8bb831800 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.945+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd23000 con 0x55e8bbbc3400 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b752ca80 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b752ca80 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b752ca80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94458071304448 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd23400 con 0x55e8bbbc3400 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify (1) cookie 94458069711616 notify 133143986179 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x55e8bbce9a00 con 0x55e8bbbc3400 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6af4ca700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 94458069711616 notifier 4404 bl.length()=166 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6af4ca700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6af4ca700 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T04:34:44.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6af4ca700 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bb831c00 con 0x55e8bbbc3400 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.0 [notify cookie 94458071304448 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== watch-notify(notify_complete (2) cookie 94458071304448 notify 133143986179 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e8bbce7040 con 0x55e8bbbc3400 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:44.970 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo2", 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "" 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:44.971 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo1", 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo2", 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:44.972 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:44.973 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:44.974 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.947+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.948+0000 7fd69c4a4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.948+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94458071302144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd23c00 con 0x55e8bb831800 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.949+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94458071302144] v31'85 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.949+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94458069711616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd96800 con 0x55e8bbbc3400 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.950+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94458069711616] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.950+0000 7fd6b752ca80 20 remove_watcher() i=0 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.950+0000 7fd6b752ca80 2 removed watcher, disabling cache 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.950+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94458069712768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd22c00 con 0x55e8bb831800 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.951+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94458069712768] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.951+0000 7fd6b752ca80 20 remove_watcher() i=1 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.951+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94458069713920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd96c00 con 0x55e8bbbc3c00 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.952+0000 7fd6b54d6700 1 -- 172.21.15.191:0/568525558 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94458069713920] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b200 con 0x55e8bbbc3c00 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.952+0000 7fd6b752ca80 20 remove_watcher() i=2 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.952+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94458069715072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd97000 con 0x55e8bbbc3400 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.953+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94458069715072] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7b440 con 0x55e8bbbc3400 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.953+0000 7fd6b752ca80 20 remove_watcher() i=3 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.953+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94458069716224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd97400 con 0x55e8bb831800 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.954+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94458069716224] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bb831800 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.954+0000 7fd6b752ca80 20 remove_watcher() i=4 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.954+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94458069717376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd97800 con 0x55e8bbbc3000 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.955+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94458069717376] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.955+0000 7fd6b752ca80 20 remove_watcher() i=5 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.955+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94458069718528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd97c00 con 0x55e8bbbc3000 2024-04-27T04:34:44.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.956+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94458069718528] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.956+0000 7fd6b752ca80 20 remove_watcher() i=6 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.956+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94458069719680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e8bbd23800 con 0x55e8bbbc3000 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b5cd7700 1 -- 172.21.15.191:0/568525558 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94458069719680] v31'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e8bbc7ab40 con 0x55e8bbbc3000 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 20 remove_watcher() i=7 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e8bbbc3000 msgr2=0x55e8bbb1b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e8bbbc3000 0x55e8bbb1b180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e8bb831800 msgr2=0x55e8bbb19b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e8bb831800 0x55e8bbb19b80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e8bbbc3400 msgr2=0x55e8bbb1ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e8bbbc3400 0x55e8bbb1ac00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e8bbbc3c00 msgr2=0x55e8bbb1a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e8bbbc3c00 0x55e8bbb1a100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2c00 msgr2=0x55e8bbb19600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2c00 0x55e8bbb19600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55e8bbb13890 tx=0x55e8bb921800 comp rx=0 tx=0).stop 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 msgr2=0x55e8bbb18b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/568525558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb831400 0x55e8bbb18b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55e8bc6ae240 tx=0x55e8bbb7cd20 comp rx=0 tx=0).stop 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 reap_dead start 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b4cd5700 1 -- 172.21.15.191:0/568525558 reap_dead start 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 shutdown_connections 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 >> 172.21.15.191:0/568525558 conn(0x55e8bb830c00 msgr2=0x55e8bb91d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 shutdown_connections 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.957+0000 7fd6b752ca80 1 -- 172.21.15.191:0/568525558 wait complete. 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbd22000 msgr2=0x55e8bbb1b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbd22000 0x55e8bbb1b700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55e8bbc29fb0 tx=0x55e8bbcab320 comp rx=0 tx=0).stop 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 msgr2=0x55e8bbb1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55e8bbc9c270 tx=0x55e8bb9218c0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 shutdown_connections 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbd22000 0x55e8bbb1b700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/3266268801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bbc87400 0x55e8bbb1a680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 >> 172.21.15.191:0/3266268801 conn(0x55e8bbc87000 msgr2=0x55e8bb91e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 shutdown_connections 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/3266268801 wait complete. 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2000 msgr2=0x55e8bbb19080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2000 0x55e8bbb19080 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55e8bc6af590 tx=0x55e8bbb7c240 comp rx=0 tx=0).stop 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 msgr2=0x55e8bbb18580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55e8bc5f7f80 tx=0x55e8bb921a40 comp rx=0 tx=0).stop 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 shutdown_connections 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e8bbbc2000 0x55e8bbb19080 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 --2- 172.21.15.191:0/2223641004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8bb830800 0x55e8bbb18580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 >> 172.21.15.191:0/2223641004 conn(0x55e8bb830400 msgr2=0x55e8bb91d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 shutdown_connections 2024-04-27T04:34:44.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:44.958+0000 7fd6b752ca80 1 -- 172.21.15.191:0/2223641004 wait complete. 2024-04-27T04:34:44.985 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:44.985 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.985 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:34:44.985 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:34:45.031 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.031 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd8fada80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55e63b69ac00 0x55e63b982580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd8fada80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e63b9c5500 con 0x55e63b69a800 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd8fada80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e63b73ca00 con 0x55e63b69ac00 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd6756700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55e63b69ac00 0x55e63b982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd6f57700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55e63b69a800 0x55e63b786800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55398/0 (socket says 172.21.15.191:55398) 2024-04-27T04:34:45.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd6f57700 1 -- 172.21.15.191:0/1495434044 learned_addr learned my addr 172.21.15.191:0/1495434044 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd6756700 1 -- 172.21.15.191:0/1495434044 >> v1:172.21.15.138:6789/0 conn(0x55e63b69a800 legacy=0x55e63b786800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd6756700 1 -- 172.21.15.191:0/1495434044 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c3d40 con 0x55e63b69ac00 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.029+0000 7f3bd5f55700 1 -- 172.21.15.191:0/1495434044 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147833540 0 0) 0x55e63c468000 con 0x55e63b69a800 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd6756700 1 --2- 172.21.15.191:0/1495434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69ac00 0x55e63b982580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e63b97d890 tx=0x55e63b78b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77150984302c310 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd5f55700 1 -- 172.21.15.191:0/1495434044 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7534a0 con 0x55e63b69ac00 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd5f55700 1 -- 172.21.15.191:0/1495434044 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63b9c5800 con 0x55e63b69ac00 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd5f55700 1 -- 172.21.15.191:0/1495434044 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b753600 con 0x55e63b69ac00 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 -- 172.21.15.191:0/1495434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69ac00 msgr2=0x55e63b982580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/1495434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69ac00 0x55e63b982580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e63b97d890 tx=0x55e63b78b920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 -- 172.21.15.191:0/1495434044 shutdown_connections 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/1495434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69ac00 0x55e63b982580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 -- 172.21.15.191:0/1495434044 >> 172.21.15.191:0/1495434044 conn(0x55e63b69a400 msgr2=0x55e63b786000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 -- 172.21.15.191:0/1495434044 shutdown_connections 2024-04-27T04:34:45.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.030+0000 7f3bd8fada80 1 -- 172.21.15.191:0/1495434044 wait complete. 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd8fada80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd8fada80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63b73c500 con 0x55e63b69a800 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd6f57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd6f57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39396/0 (socket says 172.21.15.191:39396) 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd6f57700 1 -- 172.21.15.191:0/3392954858 learned_addr learned my addr 172.21.15.191:0/3392954858 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.031+0000 7f3bd6f57700 1 -- 172.21.15.191:0/3392954858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c3ba0 con 0x55e63b69a800 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/3392954858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55e63c46b770 tx=0x55e63b78bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5f3070e9514633 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd5754700 1 -- 172.21.15.191:0/3392954858 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7534a0 con 0x55e63b69a800 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd5754700 1 -- 172.21.15.191:0/3392954858 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63b9c5b00 con 0x55e63b69a800 2024-04-27T04:34:45.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd5754700 1 -- 172.21.15.191:0/3392954858 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b753600 con 0x55e63b69a800 2024-04-27T04:34:45.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 -- 172.21.15.191:0/3392954858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 msgr2=0x55e63b982580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/3392954858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55e63c46b770 tx=0x55e63b78bb60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 -- 172.21.15.191:0/3392954858 shutdown_connections 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/3392954858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 -- 172.21.15.191:0/3392954858 >> 172.21.15.191:0/3392954858 conn(0x55e63b69a400 msgr2=0x55e63b787000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 -- 172.21.15.191:0/3392954858 shutdown_connections 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 -- 172.21.15.191:0/3392954858 wait complete. 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.032+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd8fada80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd8fada80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63b73c3c0 con 0x55e63b69a800 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd6f57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd6f57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39402/0 (socket says 172.21.15.191:39402) 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd6f57700 1 -- 172.21.15.191:0/870755766 learned_addr learned my addr 172.21.15.191:0/870755766 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd6f57700 1 -- 172.21.15.191:0/870755766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c3a00 con 0x55e63b69a800 2024-04-27T04:34:45.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55e63c46be30 tx=0x55e63b78ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd3f51700 1 -- 172.21.15.191:0/870755766 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7538c0 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd3f51700 1 -- 172.21.15.191:0/870755766 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63b73bc80 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e63b6c3860 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd3f51700 1 -- 172.21.15.191:0/870755766 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7534a0 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.033+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e63b6c36c0 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd3f51700 1 -- 172.21.15.191:0/870755766 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e63c43cb00 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd8fada80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd8fada80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63b73c000 con 0x55e63b69b400 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd3f51700 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2a000 0x55e63b983080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd3f51700 1 -- 172.21.15.191:0/870755766 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e63b715180 con 0x55e63b69a800 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd7758700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2a000 0x55e63b983080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd7758700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39404/0 (socket says 172.21.15.191:39404) 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd7758700 1 -- 172.21.15.191:0/2789031020 learned_addr learned my addr 172.21.15.191:0/2789031020 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd7758700 1 -- 172.21.15.191:0/2789031020 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c2000 con 0x55e63b69b400 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.034+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2a000 0x55e63b983080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55e63c52cf00 tx=0x55e63b9e6240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd7758700 1 --2- 172.21.15.191:0/2789031020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55e63c52c360 tx=0x55e63b9e70e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93ba0102e27d387 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd174c700 1 -- 172.21.15.191:0/2789031020 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7538c0 con 0x55e63b69b400 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd174c700 1 -- 172.21.15.191:0/2789031020 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63b9c4300 con 0x55e63b69b400 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd174c700 1 -- 172.21.15.191:0/2789031020 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7534a0 con 0x55e63b69b400 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2789031020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 msgr2=0x55e63b982b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/2789031020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55e63c52c360 tx=0x55e63b9e70e0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2789031020 shutdown_connections 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/2789031020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2789031020 >> 172.21.15.191:0/2789031020 conn(0x55e63b69ac00 msgr2=0x55e63b787800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2789031020 shutdown_connections 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2789031020 wait complete. 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.035+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd8fada80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd8fada80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63b6d6640 con 0x55e63b69b400 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd7758700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd7758700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39406/0 (socket says 172.21.15.191:39406) 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 learned_addr learned my addr 172.21.15.191:0/359453580 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c2ea0 con 0x55e63b69b400 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd7758700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55e63c52c3f0 tx=0x55e63b9e6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bcff49700 1 -- 172.21.15.191:0/359453580 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b753b80 con 0x55e63b69b400 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bcff49700 1 -- 172.21.15.191:0/359453580 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63b73bc80 con 0x55e63b69b400 2024-04-27T04:34:45.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e63b6c2820 con 0x55e63b69b400 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bcff49700 1 -- 172.21.15.191:0/359453580 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b7538c0 con 0x55e63b69b400 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.036+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e63b6c29c0 con 0x55e63b69b400 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bcff49700 1 -- 172.21.15.191:0/359453580 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e63c43cdc0 con 0x55e63b69b400 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bcff49700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2ac00 0x55e63b983600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bcff49700 1 -- 172.21.15.191:0/359453580 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e63b715340 con 0x55e63b69b400 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bd6756700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2ac00 0x55e63b983600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e63b69b800 0x55e63b983b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63b69bc00 con 0x55e63b69b800 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bd7758700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e63b69b800 0x55e63b983b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.037+0000 7f3bd6756700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2ac00 0x55e63b983600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55e63b97d890 tx=0x55e63b78b800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd7758700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e63b69b800 0x55e63b983b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 realm 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b000 con 0x55e63b69b800 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b400 con 0x55e63b69b800 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.038+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b800 con 0x55e63b69b800 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e63ba2bc00 0x55e63b984100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b000 con 0x55e63ba2bc00 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e63ba2bc00 0x55e63b984100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e63ba2bc00 0x55e63b984100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 20 searching for the correct realm 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.039+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b400 con 0x55e63ba2bc00 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6f57700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e63ba2b400 0x55e63b984c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b000 con 0x55e63ba2b400 2024-04-27T04:34:45.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6756700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e63ba2b400 0x55e63b984c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6756700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e63ba2b400 0x55e63b984c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6756700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e63ba2b000 0x55e63b985180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.040+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e63ba2b000 0x55e63b985180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e63ba2b000 0x55e63b985180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.041+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b400 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.042+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b400 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.043+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.044+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b400 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.045+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.046+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b400 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2bc00 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b400 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd8fada80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd8fada80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd8fada80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd8fada80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.047+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63b69bc00 con 0x55e63b69b800 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0400 con 0x55e63b69b800 2024-04-27T04:34:45.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0800 con 0x55e63ba2bc00 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 zone default found 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 4 Realm: () 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 zonegroup default 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0c00 con 0x55e63ba2b000 2024-04-27T04:34:45.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.048+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1000 con 0x55e63ba2bc00 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1400 con 0x55e63ba2bc00 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 20 started sync module instance, tier type = 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.049+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1800 con 0x55e63ba2b400 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.050+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.050+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94447330630400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0800 con 0x55e63ba2b400 2024-04-27T04:34:45.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.050+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1000 con 0x55e63b69b800 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.051+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94447330630400] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.051+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.051+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94447330631552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1400 con 0x55e63b69b800 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.051+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0400 con 0x55e63ba2bc00 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.052+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94447330631552] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.052+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.052+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94447330632704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0c00 con 0x55e63ba2bc00 2024-04-27T04:34:45.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.052+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63b69bc00 con 0x55e63ba2b400 2024-04-27T04:34:45.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.053+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94447330632704] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.053+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.053+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94447330633856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63ba2b800 con 0x55e63ba2b400 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.053+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63b69b800 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.054+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.054+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94447330635008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1800 con 0x55e63b69b800 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.054+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0800 con 0x55e63ba2b000 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.054+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94447330633856] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.055+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94447330635008] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.055+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.055+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94447330636160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1c00 con 0x55e63ba2b000 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.055+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1000 con 0x55e63ba2b000 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.056+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94447330636160] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.056+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.056+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94447330637312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1400 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.056+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0000 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94447330637312] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94447330638464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1800 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=0 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=1 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=2 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=3 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=4 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=5 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=6 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94447330638464] v31'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 20 add_watcher() i=7 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.057+0000 7f3bd8fada80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.058+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0800 con 0x55e63b69b800 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.058+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.058+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94447332212736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1c00 con 0x55e63b69b800 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94447332212736] v31'86 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bbdf25700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd8fada80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd8fada80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd8fada80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.059+0000 7f3bd8fada80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63b6d6780 con 0x55e63baf1400 2024-04-27T04:34:45.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bd6756700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bd6756700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39420/0 (socket says 172.21.15.191:39420) 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bd6756700 1 -- 172.21.15.191:0/2709580890 learned_addr learned my addr 172.21.15.191:0/2709580890 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bd6756700 1 -- 172.21.15.191:0/2709580890 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c2b60 con 0x55e63baf1400 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bd6756700 1 --2- 172.21.15.191:0/2709580890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55e63ba93800 tx=0x55e63b78b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0a6d3770e96425 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bbd724700 1 -- 172.21.15.191:0/2709580890 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63b753760 con 0x55e63baf1400 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bbd724700 1 -- 172.21.15.191:0/2709580890 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63bb10000 con 0x55e63baf1400 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.060+0000 7f3bbd724700 1 -- 172.21.15.191:0/2709580890 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63bb12000 con 0x55e63baf1400 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2709580890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 msgr2=0x55e63b984680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/2709580890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55e63ba93800 tx=0x55e63b78b980 comp rx=0 tx=0).stop 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2709580890 shutdown_connections 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/2709580890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2709580890 >> 172.21.15.191:0/2709580890 conn(0x55e63baf1000 msgr2=0x55e63b788000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2709580890 shutdown_connections 2024-04-27T04:34:45.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- 172.21.15.191:0/2709580890 wait complete. 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 Processor -- start 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- start start 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.061+0000 7f3bd8fada80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63b73c000 con 0x55e63baf1400 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd6756700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd6756700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39426/0 (socket says 172.21.15.191:39426) 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd6756700 1 -- 172.21.15.191:0/290208628 learned_addr learned my addr 172.21.15.191:0/290208628 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd6756700 1 -- 172.21.15.191:0/290208628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63b6c2d00 con 0x55e63baf1400 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd6756700 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55e63bb05260 tx=0x55e63b78b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bbbf21700 1 -- 172.21.15.191:0/290208628 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63bb12160 con 0x55e63baf1400 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bbbf21700 1 -- 172.21.15.191:0/290208628 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e63bb10180 con 0x55e63baf1400 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bbbf21700 1 -- 172.21.15.191:0/290208628 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e63bb122c0 con 0x55e63baf1400 2024-04-27T04:34:45.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e63b6c2340 con 0x55e63baf1400 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.062+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e63b6c3d40 con 0x55e63baf1400 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.063+0000 7f3bbbf21700 1 -- 172.21.15.191:0/290208628 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e63c43db80 con 0x55e63baf1400 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.063+0000 7f3bbbf21700 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63bb8c000 0x55e63b985700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.063+0000 7f3bbbf21700 1 -- 172.21.15.191:0/290208628 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e63b715500 con 0x55e63baf1400 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.063+0000 7f3bd7758700 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63bb8c000 0x55e63b985700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.063+0000 7f3bd8fada80 5 note: GC not initialized 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.063+0000 7f3bd7758700 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63bb8c000 0x55e63b985700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55e63ba93470 tx=0x55e63bb13200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bb971c700 20 reqs_thread_entry: start 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bd8fada80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bd8fada80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bb7718700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bb6ef6700 20 reqs_thread_entry: start 2024-04-27T04:34:45.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bd8fada80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.064+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf1800 con 0x55e63b69b800 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.065+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.065+0000 7f3bd8fada80 20 rados_obj.operate() r=0 bl.length=621 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:45.084 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo2", 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "" 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:45.085 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo1", 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo2", 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:45.086 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.087 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:45.088 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.065+0000 7f3bbdf25700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.065+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94447332212736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63baf0800 con 0x55e63b69b800 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.066+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94447332212736] v31'87 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.066+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94447330630400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8c400 con 0x55e63ba2b400 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.067+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94447330630400] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.067+0000 7f3bd8fada80 20 remove_watcher() i=0 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.067+0000 7f3bd8fada80 2 removed watcher, disabling cache 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.067+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94447330631552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8c800 con 0x55e63b69b800 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.068+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94447330631552] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.068+0000 7f3bd8fada80 20 remove_watcher() i=1 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.068+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94447330632704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8cc00 con 0x55e63ba2bc00 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.069+0000 7f3bd6f57700 1 -- 172.21.15.191:0/359453580 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94447330632704] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5200 con 0x55e63ba2bc00 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.069+0000 7f3bd8fada80 20 remove_watcher() i=2 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.069+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94447330633856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8d000 con 0x55e63ba2b400 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.070+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94447330633856] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae5440 con 0x55e63ba2b400 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.070+0000 7f3bd8fada80 20 remove_watcher() i=3 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.070+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94447330635008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8d400 con 0x55e63b69b800 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.071+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94447330635008] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63b69b800 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.071+0000 7f3bd8fada80 20 remove_watcher() i=4 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.071+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94447330636160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8d800 con 0x55e63ba2b000 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.072+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94447330636160] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.072+0000 7f3bd8fada80 20 remove_watcher() i=5 2024-04-27T04:34:45.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.072+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94447330637312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8dc00 con 0x55e63ba2b000 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.073+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94447330637312] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.073+0000 7f3bd8fada80 20 remove_watcher() i=6 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.073+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94447330638464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e63bb8c800 con 0x55e63ba2b000 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd7758700 1 -- 172.21.15.191:0/359453580 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94447330638464] v31'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e63bae4b40 con 0x55e63ba2b000 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 20 remove_watcher() i=7 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e63ba2b000 msgr2=0x55e63b985180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e63ba2b000 0x55e63b985180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e63b69b800 msgr2=0x55e63b983b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e63b69b800 0x55e63b983b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e63ba2b400 msgr2=0x55e63b984c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e63ba2b400 0x55e63b984c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e63ba2bc00 msgr2=0x55e63b984100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e63ba2bc00 0x55e63b984100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2ac00 msgr2=0x55e63b983600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2ac00 0x55e63b983600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55e63b97d890 tx=0x55e63b78b800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 msgr2=0x55e63b982b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/359453580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69b400 0x55e63b982b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55e63c52c3f0 tx=0x55e63b9e6cc0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 reap_dead start 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 shutdown_connections 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 >> 172.21.15.191:0/359453580 conn(0x55e63b69ac00 msgr2=0x55e63b787800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd6756700 1 -- 172.21.15.191:0/359453580 reap_dead start 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 shutdown_connections 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.074+0000 7f3bd8fada80 1 -- 172.21.15.191:0/359453580 wait complete. 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63bb8c000 msgr2=0x55e63b985700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63bb8c000 0x55e63b985700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55e63ba93470 tx=0x55e63bb13200 comp rx=0 tx=0).stop 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 msgr2=0x55e63b984680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55e63bb05260 tx=0x55e63b78b8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 shutdown_connections 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63bb8c000 0x55e63b985700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/290208628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63baf1400 0x55e63b984680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 >> 172.21.15.191:0/290208628 conn(0x55e63baf1000 msgr2=0x55e63b788000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 shutdown_connections 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/290208628 wait complete. 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2a000 msgr2=0x55e63b983080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2a000 0x55e63b983080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55e63c52cf00 tx=0x55e63b9e6240 comp rx=0 tx=0).stop 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 msgr2=0x55e63b982580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55e63c46be30 tx=0x55e63b78ba40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 shutdown_connections 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e63ba2a000 0x55e63b983080 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 --2- 172.21.15.191:0/870755766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e63b69a800 0x55e63b982580 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 >> 172.21.15.191:0/870755766 conn(0x55e63b69a400 msgr2=0x55e63b787000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 shutdown_connections 2024-04-27T04:34:45.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.075+0000 7f3bd8fada80 1 -- 172.21.15.191:0/870755766 wait complete. 2024-04-27T04:34:45.099 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:45.099 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-27T04:34:45.099 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-27T04:34:45.099 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-04-27T04:34:45.147 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.147 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729f162a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x560776df0c00 0x5607770d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729f162a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56077711b500 con 0x560776df0800 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729f162a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x560776e92a00 con 0x560776df0c00 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729c90b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x560776df0c00 0x5607770d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729d10c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x560776df0800 0x560776edc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55414/0 (socket says 172.21.15.191:55414) 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.145+0000 7f729d10c700 1 -- 172.21.15.191:0/4225235424 learned_addr learned my addr 172.21.15.191:0/4225235424 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729c90b700 1 -- 172.21.15.191:0/4225235424 >> v1:172.21.15.138:6789/0 conn(0x560776df0800 legacy=0x560776edc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729c90b700 1 -- 172.21.15.191:0/4225235424 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e19d40 con 0x560776df0c00 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729c90b700 1 --2- 172.21.15.191:0/4225235424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0c00 0x5607770d8580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5607770d3890 tx=0x560776ee1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a220c7fba3ddbee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729c10a700 1 -- 172.21.15.191:0/4225235424 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea94a0 con 0x560776df0c00 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729c10a700 1 -- 172.21.15.191:0/4225235424 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56077711b800 con 0x560776df0c00 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729c10a700 1 -- 172.21.15.191:0/4225235424 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea9600 con 0x560776df0c00 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 -- 172.21.15.191:0/4225235424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0c00 msgr2=0x5607770d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 --2- 172.21.15.191:0/4225235424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0c00 0x5607770d8580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5607770d3890 tx=0x560776ee1920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 -- 172.21.15.191:0/4225235424 shutdown_connections 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 --2- 172.21.15.191:0/4225235424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0c00 0x5607770d8580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 -- 172.21.15.191:0/4225235424 >> 172.21.15.191:0/4225235424 conn(0x560776df0400 msgr2=0x560776edc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 -- 172.21.15.191:0/4225235424 shutdown_connections 2024-04-27T04:34:45.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.146+0000 7f729f162a80 1 -- 172.21.15.191:0/4225235424 wait complete. 2024-04-27T04:34:45.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729f162a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729f162a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x560776e92500 con 0x560776df0800 2024-04-27T04:34:45.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729d10c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729d10c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39442/0 (socket says 172.21.15.191:39442) 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.147+0000 7f729d10c700 1 -- 172.21.15.191:0/859156394 learned_addr learned my addr 172.21.15.191:0/859156394 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729d10c700 1 -- 172.21.15.191:0/859156394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e19ba0 con 0x560776df0800 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729d10c700 1 --2- 172.21.15.191:0/859156394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x560777c5b8c0 tx=0x560776ee1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec135e74a6f2af77 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729b909700 1 -- 172.21.15.191:0/859156394 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea94a0 con 0x560776df0800 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729b909700 1 -- 172.21.15.191:0/859156394 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56077711bb00 con 0x560776df0800 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729b909700 1 -- 172.21.15.191:0/859156394 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea9600 con 0x560776df0800 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 -- 172.21.15.191:0/859156394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 msgr2=0x5607770d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 --2- 172.21.15.191:0/859156394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x560777c5b8c0 tx=0x560776ee1b60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 -- 172.21.15.191:0/859156394 shutdown_connections 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 --2- 172.21.15.191:0/859156394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 -- 172.21.15.191:0/859156394 >> 172.21.15.191:0/859156394 conn(0x560776df0400 msgr2=0x560776edd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 -- 172.21.15.191:0/859156394 shutdown_connections 2024-04-27T04:34:45.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.148+0000 7f729f162a80 1 -- 172.21.15.191:0/859156394 wait complete. 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729f162a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729f162a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x560776e923c0 con 0x560776df0800 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729d10c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729d10c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39446/0 (socket says 172.21.15.191:39446) 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729d10c700 1 -- 172.21.15.191:0/2595682440 learned_addr learned my addr 172.21.15.191:0/2595682440 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729d10c700 1 -- 172.21.15.191:0/2595682440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e19a00 con 0x560776df0800 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.149+0000 7f729d10c700 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x560777c5bb00 tx=0x560776ee1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.150+0000 7f729a106700 1 -- 172.21.15.191:0/2595682440 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea98c0 con 0x560776df0800 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.150+0000 7f729a106700 1 -- 172.21.15.191:0/2595682440 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560776e91c80 con 0x560776df0800 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.150+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560776e19860 con 0x560776df0800 2024-04-27T04:34:45.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.150+0000 7f729a106700 1 -- 172.21.15.191:0/2595682440 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea94a0 con 0x560776df0800 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.150+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560776e196c0 con 0x560776df0800 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.150+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729a106700 1 -- 172.21.15.191:0/2595682440 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560777b92b00 con 0x560776df0800 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729f162a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729f162a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x560776e2c640 con 0x560776df1400 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729a106700 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182000 0x5607770d9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729a106700 1 -- 172.21.15.191:0/2595682440 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x560776e6b180 con 0x560776df0800 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d10c700 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182000 0x5607770d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d90d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d90d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39450/0 (socket says 172.21.15.191:39450) 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d90d700 1 -- 172.21.15.191:0/35009245 learned_addr learned my addr 172.21.15.191:0/35009245 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d90d700 1 -- 172.21.15.191:0/35009245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e18000 con 0x560776df1400 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d10c700 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182000 0x5607770d9080 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x560777c6f020 tx=0x56077713c240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f729d90d700 1 --2- 172.21.15.191:0/35009245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x560777c6e3c0 tx=0x56077713d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240105c2ee3ed9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f7297901700 1 -- 172.21.15.191:0/35009245 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea98c0 con 0x560776df1400 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f7297901700 1 -- 172.21.15.191:0/35009245 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56077711a300 con 0x560776df1400 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.151+0000 7f7297901700 1 -- 172.21.15.191:0/35009245 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea94a0 con 0x560776df1400 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- 172.21.15.191:0/35009245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 msgr2=0x5607770d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 --2- 172.21.15.191:0/35009245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x560777c6e3c0 tx=0x56077713d0e0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- 172.21.15.191:0/35009245 shutdown_connections 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 --2- 172.21.15.191:0/35009245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- 172.21.15.191:0/35009245 >> 172.21.15.191:0/35009245 conn(0x560776df0c00 msgr2=0x560776edd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- 172.21.15.191:0/35009245 shutdown_connections 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- 172.21.15.191:0/35009245 wait complete. 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729f162a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x560776e2c8c0 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729d90d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729d90d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39458/0 (socket says 172.21.15.191:39458) 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.152+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 learned_addr learned my addr 172.21.15.191:0/2249665123 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e18ea0 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f729d90d700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x560777c6e240 tx=0x56077713ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f72960fe700 1 -- 172.21.15.191:0/2249665123 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea9b80 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f72960fe700 1 -- 172.21.15.191:0/2249665123 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560776e91c80 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560776e18820 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f72960fe700 1 -- 172.21.15.191:0/2249665123 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea98c0 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.153+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560776e189c0 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f72960fe700 1 -- 172.21.15.191:0/2249665123 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560777b92dc0 con 0x560776df1400 2024-04-27T04:34:45.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f72960fe700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182c00 0x5607770d9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f72960fe700 1 -- 172.21.15.191:0/2249665123 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x560776e6b340 con 0x560776df1400 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729c90b700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182c00 0x5607770d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x560776df1800 0x5607770d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560776df1c00 con 0x560776df1800 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729d90d700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x560776df1800 0x5607770d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729c90b700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182c00 0x5607770d9600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5607770d3890 tx=0x560776ee1800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.154+0000 7f729d90d700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x560776df1800 0x5607770d9b80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 realm 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183000 con 0x560776df1800 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183400 con 0x560776df1800 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183800 con 0x560776df1800 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x560777183c00 0x5607770da100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183000 con 0x560777183c00 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.155+0000 7f729d10c700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x560777183c00 0x5607770da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729d10c700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x560777183c00 0x5607770da100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729f162a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729f162a80 20 searching for the correct realm 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183400 con 0x560777183c00 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729d10c700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x560777183400 0x5607770dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183000 con 0x560777183400 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.156+0000 7f729c90b700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x560777183400 0x5607770dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.157+0000 7f729c90b700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x560777183400 0x5607770dac00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.157+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.157+0000 7f729c90b700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x560777183000 0x5607770db180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.157+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.157+0000 7f729d90d700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x560777183000 0x5607770db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.157+0000 7f729d90d700 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x560777183000 0x5607770db180 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.158+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183400 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.159+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183400 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.160+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.161+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183400 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.162+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183400 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183c00 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.163+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183400 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560776df1c00 con 0x560776df1800 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.164+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246400 con 0x560776df1800 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246800 con 0x560777183c00 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 zone default found 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 4 Realm: () 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 zonegroup default 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246c00 con 0x560777183000 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247000 con 0x560777183c00 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.165+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247400 con 0x560777183c00 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.166+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.166+0000 7f729f162a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.166+0000 7f729f162a80 20 started sync module instance, tier type = 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.166+0000 7f729f162a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.166+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247800 con 0x560777183400 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.167+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.167+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94590062099200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246800 con 0x560777183400 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.167+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247000 con 0x560776df1800 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.168+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94590062099200] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.168+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.168+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94590062100352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247400 con 0x560776df1800 2024-04-27T04:34:45.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.168+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246400 con 0x560777183c00 2024-04-27T04:34:45.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.168+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94590062100352] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.169+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.169+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94590062101504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246c00 con 0x560777183c00 2024-04-27T04:34:45.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.169+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560776df1c00 con 0x560777183400 2024-04-27T04:34:45.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.170+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94590062101504] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.170+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.170+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94590062102656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777183800 con 0x560777183400 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.170+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560776df1800 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.171+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.171+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94590062103808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247800 con 0x560776df1800 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.171+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246800 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.171+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94590062102656] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.172+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94590062103808] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.172+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.172+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94590062104960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247c00 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.172+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247000 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.173+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94590062104960] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.173+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.173+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94590062106112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247400 con 0x560777183000 2024-04-27T04:34:45.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.173+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246000 con 0x560777183000 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94590062107264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247800 con 0x560777183000 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=0 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=1 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=2 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=3 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=4 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=5 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94590062106112] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.174+0000 7f729f162a80 20 add_watcher() i=6 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.176+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94590062107264] v31'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.176+0000 7f729f162a80 20 add_watcher() i=7 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.176+0000 7f729f162a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.177+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777246800 con 0x560776df1800 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.177+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.177+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94590063689728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247c00 con 0x560776df1800 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94590063689728] v31'88 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f72840da700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729f162a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729f162a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729f162a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.178+0000 7f729f162a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x560776e923c0 con 0x560777247400 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f729c90b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f729c90b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39460/0 (socket says 172.21.15.191:39460) 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f729c90b700 1 -- 172.21.15.191:0/3926734179 learned_addr learned my addr 172.21.15.191:0/3926734179 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f729c90b700 1 -- 172.21.15.191:0/3926734179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e18b60 con 0x560777247400 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f729c90b700 1 --2- 172.21.15.191:0/3926734179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56077725c060 tx=0x560776ee1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7469ebfdda62f114 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f72838d9700 1 -- 172.21.15.191:0/3926734179 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560776ea9760 con 0x560777247400 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f72838d9700 1 -- 172.21.15.191:0/3926734179 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560777268000 con 0x560777247400 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.179+0000 7f72838d9700 1 -- 172.21.15.191:0/3926734179 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56077726a000 con 0x560777247400 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- 172.21.15.191:0/3926734179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 msgr2=0x5607770da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 --2- 172.21.15.191:0/3926734179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56077725c060 tx=0x560776ee1980 comp rx=0 tx=0).stop 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- 172.21.15.191:0/3926734179 shutdown_connections 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 --2- 172.21.15.191:0/3926734179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- 172.21.15.191:0/3926734179 >> 172.21.15.191:0/3926734179 conn(0x560777247000 msgr2=0x560776ede000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- 172.21.15.191:0/3926734179 shutdown_connections 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- 172.21.15.191:0/3926734179 wait complete. 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 Processor -- start 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- start start 2024-04-27T04:34:45.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729f162a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x560776e92500 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729c90b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729c90b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39472/0 (socket says 172.21.15.191:39472) 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.180+0000 7f729c90b700 1 -- 172.21.15.191:0/2546670226 learned_addr learned my addr 172.21.15.191:0/2546670226 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f729c90b700 1 -- 172.21.15.191:0/2546670226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560776e18d00 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f729c90b700 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x56077725c270 tx=0x560776ee18c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f72820d6700 1 -- 172.21.15.191:0/2546670226 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56077726a160 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f72820d6700 1 -- 172.21.15.191:0/2546670226 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560777268180 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f72820d6700 1 -- 172.21.15.191:0/2546670226 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56077726a2c0 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560776e18340 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.181+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560776e19d40 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.182+0000 7f72820d6700 1 -- 172.21.15.191:0/2546670226 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560777b93b80 con 0x560777247400 2024-04-27T04:34:45.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.182+0000 7f72820d6700 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5607772e2000 0x5607770db700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.182+0000 7f72820d6700 1 -- 172.21.15.191:0/2546670226 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x560776e6b500 con 0x560777247400 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.182+0000 7f729d90d700 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5607772e2000 0x5607770db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.182+0000 7f729f162a80 5 note: GC not initialized 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.182+0000 7f729d90d700 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5607772e2000 0x5607770db700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5607771dff80 tx=0x560777267200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f727f8d1700 20 reqs_thread_entry: start 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729f162a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729f162a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f727d8cd700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f727d0ab700 20 reqs_thread_entry: start 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729f162a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729f162a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247800 con 0x560776df1800 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.183+0000 7f729f162a80 20 rados_obj.operate() r=0 bl.length=621 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.184+0000 7f729f162a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.184+0000 7f729f162a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.184+0000 7f729f162a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.184+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~539 in=539b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e2400 con 0x560776df1800 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~539] v31'9 uv9 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729f162a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729f162a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729f162a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94590063692032 in=727b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e2800 con 0x560776df1800 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94590062103808 notify 133143986184 ret 0) v3 ==== 757+0+0 (crc 0 0 0) 0x5607771efd40 con 0x560776df1800 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f7297901700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986184 cookie 94590062103808 notifier 4420 bl.length()=715 2024-04-27T04:34:45.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94590063692032 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f7297901700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f7297901700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f7297901700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777356000 con 0x560776df1800 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94590063692032 notify 133143986184 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5607772adba0 con 0x560776df1800 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.185+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e2c00 con 0x560776df1800 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.186+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(66 foo:foo1 [delete] v31'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.186+0000 7f729f162a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.186+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94590063692032 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e3000 con 0x560777183400 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.187+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== watch-notify(notify (1) cookie 94590062099200 notify 133143986180 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x5607772aba00 con 0x560777183400 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.187+0000 7f7297901700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94590062099200 notifier 4420 bl.length()=159 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.187+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.0 [notify cookie 94590063692032 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.187+0000 7f7297901700 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777356400 con 0x560777183400 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.187+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify_complete (2) cookie 94590063692032 notify 133143986180 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5607772a9040 con 0x560777183400 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.187+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:45.205 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo1", 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "full-control" 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo2", 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "" 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:45.206 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo2", 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:45.209 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:45.209 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:45.209 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:45.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.188+0000 7f72840da700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.188+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94590063689728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e3400 con 0x560776df1800 2024-04-27T04:34:45.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.189+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94590063689728] v31'89 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.189+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94590062099200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e3800 con 0x560777183400 2024-04-27T04:34:45.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.190+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94590062099200] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.190+0000 7f729f162a80 20 remove_watcher() i=0 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.190+0000 7f729f162a80 2 removed watcher, disabling cache 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.190+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94590062100352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e3c00 con 0x560776df1800 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.190+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94590062100352] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.191+0000 7f729f162a80 20 remove_watcher() i=1 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.191+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94590062101504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e3000 con 0x560777183c00 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.191+0000 7f729d10c700 1 -- 172.21.15.191:0/2249665123 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94590062101504] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b200 con 0x560777183c00 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.191+0000 7f729f162a80 20 remove_watcher() i=2 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.191+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94590062102656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e3800 con 0x560777183400 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.192+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94590062102656] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723b440 con 0x560777183400 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.192+0000 7f729f162a80 20 remove_watcher() i=3 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.192+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94590062103808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e2400 con 0x560776df1800 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.193+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94590062103808] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560776df1800 2024-04-27T04:34:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.193+0000 7f729f162a80 20 remove_watcher() i=4 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.193+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94590062104960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e2800 con 0x560777183000 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.194+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94590062104960] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.194+0000 7f729f162a80 20 remove_watcher() i=5 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.194+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94590062106112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5607772e2c00 con 0x560777183000 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.195+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94590062106112] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.195+0000 7f729f162a80 20 remove_watcher() i=6 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.195+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94590062107264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560777247c00 con 0x560777183000 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729d90d700 1 -- 172.21.15.191:0/2249665123 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94590062107264] v31'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56077723ab40 con 0x560777183000 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 20 remove_watcher() i=7 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x560777183000 msgr2=0x5607770db180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x560777183000 0x5607770db180 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x560776df1800 msgr2=0x5607770d9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x560776df1800 0x5607770d9b80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x560777183400 msgr2=0x5607770dac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x560777183400 0x5607770dac00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x560777183c00 msgr2=0x5607770da100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x560777183c00 0x5607770da100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182c00 msgr2=0x5607770d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182c00 0x5607770d9600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5607770d3890 tx=0x560776ee1800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 msgr2=0x5607770d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 --2- 172.21.15.191:0/2249665123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df1400 0x5607770d8b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x560777c6e240 tx=0x56077713ccc0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 reap_dead start 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729c90b700 1 -- 172.21.15.191:0/2249665123 reap_dead start 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 shutdown_connections 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 >> 172.21.15.191:0/2249665123 conn(0x560776df0c00 msgr2=0x560776edd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 shutdown_connections 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.196+0000 7f729f162a80 1 -- 172.21.15.191:0/2249665123 wait complete. 2024-04-27T04:34:45.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5607772e2000 msgr2=0x5607770db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5607772e2000 0x5607770db700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5607771dff80 tx=0x560777267200 comp rx=0 tx=0).stop 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 msgr2=0x5607770da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x56077725c270 tx=0x560776ee18c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 shutdown_connections 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5607772e2000 0x5607770db700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2546670226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777247400 0x5607770da680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 >> 172.21.15.191:0/2546670226 conn(0x560777247000 msgr2=0x560776ede000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 shutdown_connections 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2546670226 wait complete. 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182000 msgr2=0x5607770d9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182000 0x5607770d9080 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x560777c6f020 tx=0x56077713c240 comp rx=0 tx=0).stop 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 msgr2=0x5607770d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x560777c5bb00 tx=0x560776ee1a40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 shutdown_connections 2024-04-27T04:34:45.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x560777182000 0x5607770d9080 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 --2- 172.21.15.191:0/2595682440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560776df0800 0x5607770d8580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 >> 172.21.15.191:0/2595682440 conn(0x560776df0400 msgr2=0x560776edd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 shutdown_connections 2024-04-27T04:34:45.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.197+0000 7f729f162a80 1 -- 172.21.15.191:0/2595682440 wait complete. 2024-04-27T04:34:45.219 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:45.219 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-27T04:34:45.219 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-27T04:34:45.220 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-04-27T04:34:45.265 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.266 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.263+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01df509a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5627a7676c00 0x5627a795e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01df509a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5627a79a1500 con 0x5627a7676800 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01df509a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5627a7718a00 con 0x5627a7676c00 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dccb2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5627a7676c00 0x5627a795e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dd4b3700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5627a7676800 0x5627a7762800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55422/0 (socket says 172.21.15.191:55422) 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dccb2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5627a7676c00 0x5627a795e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39480/0 (socket says 172.21.15.191:39480) 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dd4b3700 1 -- 172.21.15.191:0/1939002146 learned_addr learned my addr 172.21.15.191:0/1939002146 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dccb2700 1 -- 172.21.15.191:0/1939002146 >> v1:172.21.15.138:6789/0 conn(0x5627a7676800 legacy=0x5627a7762800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dccb2700 1 -- 172.21.15.191:0/1939002146 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769fd40 con 0x5627a7676c00 2024-04-27T04:34:45.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.264+0000 7f01dccb2700 1 --2- 172.21.15.191:0/1939002146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676c00 0x5627a795e580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5627a7959890 tx=0x5627a7767920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88696f41e26b468c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01dc4b1700 1 -- 172.21.15.191:0/1939002146 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f4a0 con 0x5627a7676c00 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01dc4b1700 1 -- 172.21.15.191:0/1939002146 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a79a1800 con 0x5627a7676c00 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01dc4b1700 1 -- 172.21.15.191:0/1939002146 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f600 con 0x5627a7676c00 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 -- 172.21.15.191:0/1939002146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676c00 msgr2=0x5627a795e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 --2- 172.21.15.191:0/1939002146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676c00 0x5627a795e580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5627a7959890 tx=0x5627a7767920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 -- 172.21.15.191:0/1939002146 shutdown_connections 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 --2- 172.21.15.191:0/1939002146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676c00 0x5627a795e580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 -- 172.21.15.191:0/1939002146 >> 172.21.15.191:0/1939002146 conn(0x5627a7676400 msgr2=0x5627a7762000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 -- 172.21.15.191:0/1939002146 shutdown_connections 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.265+0000 7f01df509a80 1 -- 172.21.15.191:0/1939002146 wait complete. 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01df509a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01df509a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a7718500 con 0x5627a7676800 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39494/0 (socket says 172.21.15.191:39494) 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dd4b3700 1 -- 172.21.15.191:0/4291477411 learned_addr learned my addr 172.21.15.191:0/4291477411 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dd4b3700 1 -- 172.21.15.191:0/4291477411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769fba0 con 0x5627a7676800 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/4291477411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5627a843d800 tx=0x5627a7767b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8550ec8a727193d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dbcb0700 1 -- 172.21.15.191:0/4291477411 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f4a0 con 0x5627a7676800 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dbcb0700 1 -- 172.21.15.191:0/4291477411 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a79a1b00 con 0x5627a7676800 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.266+0000 7f01dbcb0700 1 -- 172.21.15.191:0/4291477411 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f600 con 0x5627a7676800 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- 172.21.15.191:0/4291477411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 msgr2=0x5627a795e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 --2- 172.21.15.191:0/4291477411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5627a843d800 tx=0x5627a7767b60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- 172.21.15.191:0/4291477411 shutdown_connections 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 --2- 172.21.15.191:0/4291477411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- 172.21.15.191:0/4291477411 >> 172.21.15.191:0/4291477411 conn(0x5627a7676400 msgr2=0x5627a7763000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- 172.21.15.191:0/4291477411 shutdown_connections 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- 172.21.15.191:0/4291477411 wait complete. 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.267+0000 7f01df509a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a77183c0 con 0x5627a7676800 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39502/0 (socket says 172.21.15.191:39502) 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01dd4b3700 1 -- 172.21.15.191:0/3263538389 learned_addr learned my addr 172.21.15.191:0/3263538389 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01dd4b3700 1 -- 172.21.15.191:0/3263538389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769fa00 con 0x5627a7676800 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5627a843d860 tx=0x5627a7767a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01da4ad700 1 -- 172.21.15.191:0/3263538389 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f8c0 con 0x5627a7676800 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01da4ad700 1 -- 172.21.15.191:0/3263538389 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a7717c80 con 0x5627a7676800 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627a769f860 con 0x5627a7676800 2024-04-27T04:34:45.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01da4ad700 1 -- 172.21.15.191:0/3263538389 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f4a0 con 0x5627a7676800 2024-04-27T04:34:45.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.268+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627a769f6c0 con 0x5627a7676800 2024-04-27T04:34:45.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01da4ad700 1 -- 172.21.15.191:0/3263538389 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5627a8418b00 con 0x5627a7676800 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01da4ad700 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7677400 0x5627a795f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01da4ad700 1 -- 172.21.15.191:0/3263538389 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5627a76f1180 con 0x5627a7676800 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7677400 0x5627a795f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01df509a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01df509a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a76b2640 con 0x5627a7677800 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39514/0 (socket says 172.21.15.191:39514) 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01dd4b3700 1 -- 172.21.15.191:0/2357485127 learned_addr learned my addr 172.21.15.191:0/2357485127 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7677400 0x5627a795f080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5627a7a06030 tx=0x5627a79bcde0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.269+0000 7f01dd4b3700 1 -- 172.21.15.191:0/2357485127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769e000 con 0x5627a7677800 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/2357485127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5627a7a06870 tx=0x5627a79bc240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f5a2f3e96451c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01d7ca8700 1 -- 172.21.15.191:0/2357485127 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f600 con 0x5627a7677800 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01d7ca8700 1 -- 172.21.15.191:0/2357485127 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a79a0300 con 0x5627a7677800 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01d7ca8700 1 -- 172.21.15.191:0/2357485127 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772fa20 con 0x5627a7677800 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 -- 172.21.15.191:0/2357485127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 msgr2=0x5627a795eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 --2- 172.21.15.191:0/2357485127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5627a7a06870 tx=0x5627a79bc240 comp rx=0 tx=0).stop 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 -- 172.21.15.191:0/2357485127 shutdown_connections 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 --2- 172.21.15.191:0/2357485127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 -- 172.21.15.191:0/2357485127 >> 172.21.15.191:0/2357485127 conn(0x5627a7676c00 msgr2=0x5627a7763800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 -- 172.21.15.191:0/2357485127 shutdown_connections 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 -- 172.21.15.191:0/2357485127 wait complete. 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.270+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01df509a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01df509a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a76b28c0 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01dd4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39520/0 (socket says 172.21.15.191:39520) 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 learned_addr learned my addr 172.21.15.191:0/860669799 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769eea0 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5627a7a06b40 tx=0x5627a79bcfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01d64a5700 1 -- 172.21.15.191:0/860669799 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f600 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01d64a5700 1 -- 172.21.15.191:0/860669799 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a7717e00 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627a769e820 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01d64a5700 1 -- 172.21.15.191:0/860669799 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772fa20 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.271+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627a769e9c0 con 0x5627a7677800 2024-04-27T04:34:45.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.272+0000 7f01d64a5700 1 -- 172.21.15.191:0/860669799 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5627a8418dc0 con 0x5627a7677800 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.272+0000 7f01d64a5700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a84eac00 0x5627a795f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.272+0000 7f01d64a5700 1 -- 172.21.15.191:0/860669799 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5627a76f1340 con 0x5627a7677800 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.272+0000 7f01dccb2700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a84eac00 0x5627a795f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.272+0000 7f01dccb2700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a84eac00 0x5627a795f600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5627a7959890 tx=0x5627a7767800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627a7677c00 0x5627a795fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84eb000 con 0x5627a7677c00 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627a7677c00 0x5627a795fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627a7677c00 0x5627a795fb80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 20 realm 2024-04-27T04:34:45.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.273+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84eb400 con 0x5627a7677c00 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84eb800 con 0x5627a7677c00 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84ebc00 con 0x5627a7677c00 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627a84eb000 0x5627a7960100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84eb400 con 0x5627a84eb000 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627a84eb000 0x5627a7960100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.274+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627a84eb000 0x5627a7960100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01df509a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01df509a80 20 searching for the correct realm 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e000 con 0x5627a84eb000 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01dd4b3700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627a7a9e000 0x5627a7960c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84eb400 con 0x5627a7a9e000 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01dccb2700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627a7a9e000 0x5627a7960c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.275+0000 7f01dccb2700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627a7a9e000 0x5627a7960c00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01dccb2700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627a84eb400 0x5627a7961180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627a84eb400 0x5627a7961180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627a84eb400 0x5627a7961180 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.276+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7a9e000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.277+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.278+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7a9e000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.279+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.280+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7a9e000 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.281+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7a9e000 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.282+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb400 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7677c00 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a7a9e000 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01df509a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01df509a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01df509a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01df509a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.283+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e400 con 0x5627a7677c00 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e800 con 0x5627a7677c00 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9ec00 con 0x5627a84eb000 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 zone default found 2024-04-27T04:34:45.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 4 Realm: () 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 zonegroup default 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f000 con 0x5627a84eb400 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.284+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f400 con 0x5627a84eb000 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e400 con 0x5627a84eb000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 20 started sync module instance, tier type = 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.285+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e800 con 0x5627a7a9e000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.286+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.286+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94728315296512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f000 con 0x5627a7a9e000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.286+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9fc00 con 0x5627a7677c00 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.287+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94728315296512] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.287+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.287+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94728315297664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84ebc00 con 0x5627a7677c00 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.287+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84eb800 con 0x5627a84eb000 2024-04-27T04:34:45.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.288+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94728315297664] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.288+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.288+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94728315298816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f800 con 0x5627a84eb000 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.288+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e800 con 0x5627a7a9e000 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.289+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94728315298816] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.289+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.289+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94728315299968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f000 con 0x5627a7a9e000 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.289+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9ec00 con 0x5627a7677c00 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.290+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.290+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94728315301120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f400 con 0x5627a7677c00 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.290+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e400 con 0x5627a84eb400 2024-04-27T04:34:45.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.290+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94728315299968] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.291+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.291+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94728315302272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84ea400 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.291+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94728315301120] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.291+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aae000 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.292+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94728315302272] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.292+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.292+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94728315303424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aae400 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.292+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aae800 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94728315304576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aaec00 con 0x5627a84eb400 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 20 add_watcher() i=0 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 20 add_watcher() i=1 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 20 add_watcher() i=2 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 20 add_watcher() i=3 2024-04-27T04:34:45.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 20 add_watcher() i=4 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.293+0000 7f01df509a80 20 add_watcher() i=5 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.294+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94728315303424] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.294+0000 7f01df509a80 20 add_watcher() i=6 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.295+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94728315304576] v31'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.295+0000 7f01df509a80 20 add_watcher() i=7 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.295+0000 7f01df509a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.295+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aaf000 con 0x5627a7677c00 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.295+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.295+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94728316723200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aaf400 con 0x5627a7677c00 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.296+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94728316723200] v31'90 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.296+0000 7f01c4481700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.296+0000 7f01df509a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.296+0000 7f01df509a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.296+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.296+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01df509a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01df509a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a77183c0 con 0x5627a7aafc00 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01dccb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01dccb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39522/0 (socket says 172.21.15.191:39522) 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01dccb2700 1 -- 172.21.15.191:0/1016792239 learned_addr learned my addr 172.21.15.191:0/1016792239 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01dccb2700 1 -- 172.21.15.191:0/1016792239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769eb60 con 0x5627a7aafc00 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01dccb2700 1 --2- 172.21.15.191:0/1016792239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5627a7ab8660 tx=0x5627a7767980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fa5f02c5001124 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01c3c80700 1 -- 172.21.15.191:0/1016792239 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a772f760 con 0x5627a7aafc00 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01c3c80700 1 -- 172.21.15.191:0/1016792239 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a7ac6000 con 0x5627a7aafc00 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.297+0000 7f01c3c80700 1 -- 172.21.15.191:0/1016792239 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a7ab0160 con 0x5627a7aafc00 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 -- 172.21.15.191:0/1016792239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 msgr2=0x5627a7960680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 --2- 172.21.15.191:0/1016792239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5627a7ab8660 tx=0x5627a7767980 comp rx=0 tx=0).stop 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 -- 172.21.15.191:0/1016792239 shutdown_connections 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 --2- 172.21.15.191:0/1016792239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 -- 172.21.15.191:0/1016792239 >> 172.21.15.191:0/1016792239 conn(0x5627a7aaf800 msgr2=0x5627a7764000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 -- 172.21.15.191:0/1016792239 shutdown_connections 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 -- 172.21.15.191:0/1016792239 wait complete. 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 Processor -- start 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.298+0000 7f01df509a80 1 -- start start 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01df509a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01df509a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a7718500 con 0x5627a7aafc00 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01dccb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01dccb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39538/0 (socket says 172.21.15.191:39538) 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01dccb2700 1 -- 172.21.15.191:0/44934809 learned_addr learned my addr 172.21.15.191:0/44934809 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01dccb2700 1 -- 172.21.15.191:0/44934809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a769ed00 con 0x5627a7aafc00 2024-04-27T04:34:45.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01dccb2700 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5627a7ab8870 tx=0x5627a77678c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01c247d700 1 -- 172.21.15.191:0/44934809 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a7ab02c0 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01c247d700 1 -- 172.21.15.191:0/44934809 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627a7ac6180 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627a769e340 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01c247d700 1 -- 172.21.15.191:0/44934809 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627a7ab0420 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.299+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627a769fd40 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.300+0000 7f01c247d700 1 -- 172.21.15.191:0/44934809 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5627a8419b80 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.300+0000 7f01c247d700 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7a9fc00 0x5627a7961700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.300+0000 7f01c247d700 1 -- 172.21.15.191:0/44934809 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5627a76f16c0 con 0x5627a7aafc00 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.300+0000 7f01df509a80 5 note: GC not initialized 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.300+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7a9fc00 0x5627a7961700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.300+0000 7f01ddcb4700 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7a9fc00 0x5627a7961700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5627a85049f0 tx=0x5627a7acd2c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01bfc78700 20 reqs_thread_entry: start 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01df509a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01df509a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01bdc74700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01bd452700 20 reqs_thread_entry: start 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01df509a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01df509a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.301+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84ea400 con 0x5627a7677c00 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.302+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~539 out=539b,getxattrs out=75b] v0'0 uv9 ondisk = 0) v8 ==== 231+0+662 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.302+0000 7f01df509a80 20 rados_obj.operate() r=0 bl.length=539 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.302+0000 7f01df509a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.302+0000 7f01df509a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.302+0000 7f01df509a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.302+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9f400 con 0x5627a7677c00 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.303+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v31'10 uv10 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.303+0000 7f01df509a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.303+0000 7f01df509a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.303+0000 7f01df509a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.303+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94728316725504 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a9e400 con 0x5627a7677c00 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94728315301120 notify 133143986185 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x5627a7baa000 con 0x5627a7677c00 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01d7ca8700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986185 cookie 94728315301120 notifier 4428 bl.length()=689 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94728316725504 out=8b] v0'0 uv75 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01d7ca8700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01d7ca8700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01d7ca8700 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aae000 con 0x5627a7677c00 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.304+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94728316725504 notify 133143986185 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5627a7baa1a0 con 0x5627a7677c00 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [ 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "foo:foo2", 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: "permissions": "" 2024-04-27T04:34:45.323 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [ 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo:foo2", 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.324 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:45.325 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:45.326 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:45.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.305+0000 7f01c4481700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.305+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94728316723200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a84ea400 con 0x5627a7677c00 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.305+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94728316723200] v31'91 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.305+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94728315296512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aae800 con 0x5627a7a9e000 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.307+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94728315296512] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.307+0000 7f01df509a80 20 remove_watcher() i=0 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.307+0000 7f01df509a80 2 removed watcher, disabling cache 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.307+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94728315297664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aaec00 con 0x5627a7677c00 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.308+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94728315297664] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.308+0000 7f01df509a80 20 remove_watcher() i=1 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.308+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94728315298816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aaf000 con 0x5627a84eb000 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.309+0000 7f01dd4b3700 1 -- 172.21.15.191:0/860669799 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94728315298816] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96fc0 con 0x5627a84eb000 2024-04-27T04:34:45.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.309+0000 7f01df509a80 20 remove_watcher() i=2 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.309+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94728315299968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7aaf400 con 0x5627a7a9e000 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.310+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94728315299968] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a97200 con 0x5627a7a9e000 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.310+0000 7f01df509a80 20 remove_watcher() i=3 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.310+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94728315301120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a90000 con 0x5627a7677c00 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.311+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94728315301120] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a7677c00 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.311+0000 7f01df509a80 20 remove_watcher() i=4 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.311+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94728315302272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a90400 con 0x5627a84eb400 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.312+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94728315302272] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.312+0000 7f01df509a80 20 remove_watcher() i=5 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.312+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94728315303424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a90800 con 0x5627a84eb400 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.313+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94728315303424] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.313+0000 7f01df509a80 20 remove_watcher() i=6 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.313+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94728315304576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5627a7a90c00 con 0x5627a84eb400 2024-04-27T04:34:45.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01ddcb4700 1 -- 172.21.15.191:0/860669799 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94728315304576] v31'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627a7a96900 con 0x5627a84eb400 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 20 remove_watcher() i=7 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627a84eb400 msgr2=0x5627a7961180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5627a84eb400 0x5627a7961180 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627a7677c00 msgr2=0x5627a795fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5627a7677c00 0x5627a795fb80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627a7a9e000 msgr2=0x5627a7960c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5627a7a9e000 0x5627a7960c00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627a84eb000 msgr2=0x5627a7960100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5627a84eb000 0x5627a7960100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a84eac00 msgr2=0x5627a795f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a84eac00 0x5627a795f600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5627a7959890 tx=0x5627a7767800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 msgr2=0x5627a795eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/860669799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7677800 0x5627a795eb00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5627a7a06b40 tx=0x5627a79bcfc0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 reap_dead start 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01dccb2700 1 -- 172.21.15.191:0/860669799 reap_dead start 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 shutdown_connections 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 >> 172.21.15.191:0/860669799 conn(0x5627a7676c00 msgr2=0x5627a7763800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 shutdown_connections 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/860669799 wait complete. 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7a9fc00 msgr2=0x5627a7961700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7a9fc00 0x5627a7961700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5627a85049f0 tx=0x5627a7acd2c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 msgr2=0x5627a7960680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5627a7ab8870 tx=0x5627a77678c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 shutdown_connections 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7a9fc00 0x5627a7961700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 --2- 172.21.15.191:0/44934809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7aafc00 0x5627a7960680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 >> 172.21.15.191:0/44934809 conn(0x5627a7aaf800 msgr2=0x5627a7764000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 shutdown_connections 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.314+0000 7f01df509a80 1 -- 172.21.15.191:0/44934809 wait complete. 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7677400 msgr2=0x5627a795f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7677400 0x5627a795f080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5627a7a06030 tx=0x5627a79bcde0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 msgr2=0x5627a795e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5627a843d860 tx=0x5627a7767a40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 shutdown_connections 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5627a7677400 0x5627a795f080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 --2- 172.21.15.191:0/3263538389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627a7676800 0x5627a795e580 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 >> 172.21.15.191:0/3263538389 conn(0x5627a7676400 msgr2=0x5627a7763000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 shutdown_connections 2024-04-27T04:34:45.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.315+0000 7f01df509a80 1 -- 172.21.15.191:0/3263538389 wait complete. 2024-04-27T04:34:45.336 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:45.337 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-27T04:34:45.337 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-27T04:34:45.337 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-04-27T04:34:45.385 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.385 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c57f97a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c57f97a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a960bad500 con 0x55a960882c00 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c57f97a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a960924a00 con 0x55a960882800 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c55f41700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c55740700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55a960882c00 0x55a96096e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55428/0 (socket says 172.21.15.191:55428) 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.383+0000 7f7c55740700 1 -- 172.21.15.191:0/764013265 learned_addr learned my addr 172.21.15.191:0/764013265 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c55f41700 1 -- 172.21.15.191:0/764013265 >> v1:172.21.15.138:6789/0 conn(0x55a960882c00 legacy=0x55a96096e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c55f41700 1 -- 172.21.15.191:0/764013265 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608abd40 con 0x55a960882800 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c55f41700 1 --2- 172.21.15.191:0/764013265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55a960b659e0 tx=0x55a960973920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cc2190a42f5cac server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c54f3f700 1 -- 172.21.15.191:0/764013265 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b4a0 con 0x55a960882800 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c54f3f700 1 -- 172.21.15.191:0/764013265 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960bad800 con 0x55a960882800 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c54f3f700 1 -- 172.21.15.191:0/764013265 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b600 con 0x55a960882800 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 -- 172.21.15.191:0/764013265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 msgr2=0x55a960b6a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/764013265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55a960b659e0 tx=0x55a960973920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 -- 172.21.15.191:0/764013265 shutdown_connections 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/764013265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 -- 172.21.15.191:0/764013265 >> 172.21.15.191:0/764013265 conn(0x55a960882400 msgr2=0x55a96096e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 -- 172.21.15.191:0/764013265 shutdown_connections 2024-04-27T04:34:45.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.384+0000 7f7c57f97a80 1 -- 172.21.15.191:0/764013265 wait complete. 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c57f97a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c57f97a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a960924500 con 0x55a960882800 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c55f41700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c55f41700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39556/0 (socket says 172.21.15.191:39556) 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.385+0000 7f7c55f41700 1 -- 172.21.15.191:0/990325837 learned_addr learned my addr 172.21.15.191:0/990325837 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c55f41700 1 -- 172.21.15.191:0/990325837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608abba0 con 0x55a960882800 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c55f41700 1 --2- 172.21.15.191:0/990325837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55a96165b740 tx=0x55a960973800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8298806449ca9a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c5473e700 1 -- 172.21.15.191:0/990325837 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b760 con 0x55a960882800 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c5473e700 1 -- 172.21.15.191:0/990325837 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960badb00 con 0x55a960882800 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c5473e700 1 -- 172.21.15.191:0/990325837 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b4a0 con 0x55a960882800 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 -- 172.21.15.191:0/990325837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 msgr2=0x55a960b6a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/990325837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55a96165b740 tx=0x55a960973800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 -- 172.21.15.191:0/990325837 shutdown_connections 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/990325837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 -- 172.21.15.191:0/990325837 >> 172.21.15.191:0/990325837 conn(0x55a960882400 msgr2=0x55a96096f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 -- 172.21.15.191:0/990325837 shutdown_connections 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 -- 172.21.15.191:0/990325837 wait complete. 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.386+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c57f97a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c57f97a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9609243c0 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c55f41700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c55f41700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39560/0 (socket says 172.21.15.191:39560) 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c55f41700 1 -- 172.21.15.191:0/4032190451 learned_addr learned my addr 172.21.15.191:0/4032190451 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c55f41700 1 -- 172.21.15.191:0/4032190451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608aba00 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c55f41700 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55a96165b5f0 tx=0x55a960973920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c52f3b700 1 -- 172.21.15.191:0/4032190451 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b600 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c52f3b700 1 -- 172.21.15.191:0/4032190451 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960badc80 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a9608ab860 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c52f3b700 1 -- 172.21.15.191:0/4032190451 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b8c0 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.387+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9608ab6c0 con 0x55a960882800 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c52f3b700 1 -- 172.21.15.191:0/4032190451 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a961624b00 con 0x55a960882800 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c57f97a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c57f97a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9608be640 con 0x55a960883400 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c52f3b700 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0a000 0x55a960b6b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c52f3b700 1 -- 172.21.15.191:0/4032190451 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a9608fd180 con 0x55a960882800 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c56742700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c56742700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39576/0 (socket says 172.21.15.191:39576) 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c56742700 1 -- 172.21.15.191:0/3891943623 learned_addr learned my addr 172.21.15.191:0/3891943623 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c55f41700 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0a000 0x55a960b6b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.388+0000 7f7c56742700 1 -- 172.21.15.191:0/3891943623 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608aa000 con 0x55a960883400 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c55f41700 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0a000 0x55a960b6b080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55a96170aed0 tx=0x55a960bd0c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c56742700 1 --2- 172.21.15.191:0/3891943623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55a96170a270 tx=0x55a960bd0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffc27ca0742f228 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c50736700 1 -- 172.21.15.191:0/3891943623 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b600 con 0x55a960883400 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c50736700 1 -- 172.21.15.191:0/3891943623 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960923c80 con 0x55a960883400 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c50736700 1 -- 172.21.15.191:0/3891943623 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b8c0 con 0x55a960883400 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3891943623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 msgr2=0x55a960b6ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3891943623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55a96170a270 tx=0x55a960bd0b40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3891943623 shutdown_connections 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3891943623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3891943623 >> 172.21.15.191:0/3891943623 conn(0x55a960882c00 msgr2=0x55a96096f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3891943623 shutdown_connections 2024-04-27T04:34:45.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3891943623 wait complete. 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.389+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c57f97a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c57f97a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9608be8c0 con 0x55a960883400 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c56742700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c56742700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39580/0 (socket says 172.21.15.191:39580) 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 learned_addr learned my addr 172.21.15.191:0/3529057430 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608aaea0 con 0x55a960883400 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c56742700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55a96170a840 tx=0x55a960bd0a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c4ef33700 1 -- 172.21.15.191:0/3529057430 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b760 con 0x55a960883400 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c4ef33700 1 -- 172.21.15.191:0/3529057430 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960923b00 con 0x55a960883400 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a9608aa820 con 0x55a960883400 2024-04-27T04:34:45.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c4ef33700 1 -- 172.21.15.191:0/3529057430 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a96093b600 con 0x55a960883400 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.390+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9608aa9c0 con 0x55a960883400 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c4ef33700 1 -- 172.21.15.191:0/3529057430 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a961624dc0 con 0x55a960883400 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c4ef33700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0ac00 0x55a960b6b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c4ef33700 1 -- 172.21.15.191:0/3529057430 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a9608fd340 con 0x55a960883400 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c55740700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0ac00 0x55a960b6b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a960883800 0x55a960b6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.391+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960883c00 con 0x55a960883800 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c56742700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a960883800 0x55a960b6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c55740700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0ac00 0x55a960b6b600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55a960c0fa70 tx=0x55a960cc00c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c56742700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a960883800 0x55a960b6bb80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 20 realm 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b000 con 0x55a960883800 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.392+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b400 con 0x55a960883800 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b800 con 0x55a960883800 2024-04-27T04:34:45.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a960c0bc00 0x55a960b6c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b000 con 0x55a960c0bc00 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c55f41700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a960c0bc00 0x55a960b6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c55f41700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a960c0bc00 0x55a960b6c100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 20 searching for the correct realm 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.393+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b400 con 0x55a960c0bc00 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55f41700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a960c0b400 0x55a960b6cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b000 con 0x55a960c0b400 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55740700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a960c0b400 0x55a960b6cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55740700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a960c0b400 0x55a960b6cc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55740700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a960c0b000 0x55a960b6d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.394+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.395+0000 7f7c56742700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a960c0b000 0x55a960b6d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.395+0000 7f7c56742700 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a960c0b000 0x55a960b6d180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.395+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.395+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.395+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.395+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b400 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.396+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b400 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.397+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.398+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b400 2024-04-27T04:34:45.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.399+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.400+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b400 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0bc00 2024-04-27T04:34:45.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.401+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b400 2024-04-27T04:34:45.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960883c00 con 0x55a960883800 2024-04-27T04:34:45.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4400 con 0x55a960883800 2024-04-27T04:34:45.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4800 con 0x55a960c0bc00 2024-04-27T04:34:45.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 zone default found 2024-04-27T04:34:45.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 4 Realm: () 2024-04-27T04:34:45.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 zonegroup default 2024-04-27T04:34:45.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.402+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4c00 con 0x55a960c0b000 2024-04-27T04:34:45.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5000 con 0x55a960c0bc00 2024-04-27T04:34:45.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5400 con 0x55a960c0bc00 2024-04-27T04:34:45.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 started sync module instance, tier type = 2024-04-27T04:34:45.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.403+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5800 con 0x55a960c0b400 2024-04-27T04:34:45.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.404+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.404+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94185960381184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4800 con 0x55a960c0b400 2024-04-27T04:34:45.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.404+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5000 con 0x55a960883800 2024-04-27T04:34:45.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.405+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94185960381184] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.405+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.405+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94185960382336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5400 con 0x55a960883800 2024-04-27T04:34:45.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.405+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4400 con 0x55a960c0bc00 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.406+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94185960382336] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.406+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.406+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94185960383488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4c00 con 0x55a960c0bc00 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.406+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960883c00 con 0x55a960c0b400 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.407+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94185960383488] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.407+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.407+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94185960384640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960c0b800 con 0x55a960c0b400 2024-04-27T04:34:45.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.407+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960883800 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.408+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94185960384640] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.410+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.410+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94185960385792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5800 con 0x55a960883800 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.410+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4800 con 0x55a960c0b000 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.411+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94185960385792] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.411+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.411+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94185960386944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5c00 con 0x55a960c0b000 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.411+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5000 con 0x55a960c0b000 2024-04-27T04:34:45.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.412+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94185960386944] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.412+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.412+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94185960388096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5400 con 0x55a960c0b000 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.412+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4000 con 0x55a960c0b000 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94185960389248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5800 con 0x55a960c0b000 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=0 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=1 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=2 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=3 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=4 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=5 2024-04-27T04:34:45.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94185960388096] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.413+0000 7f7c57f97a80 20 add_watcher() i=6 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.414+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94185960389248] v31'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.414+0000 7f7c57f97a80 20 add_watcher() i=7 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.414+0000 7f7c57f97a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.414+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd4800 con 0x55a960883800 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.414+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.414+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94185961947136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5c00 con 0x55a960883800 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94185961947136] v31'92 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c3cf0f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c57f97a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c57f97a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c57f97a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.415+0000 7f7c57f97a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9609243c0 con 0x55a960cd5400 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c55740700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c55740700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39584/0 (socket says 172.21.15.191:39584) 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c55740700 1 -- 172.21.15.191:0/1849952824 learned_addr learned my addr 172.21.15.191:0/1849952824 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c55740700 1 -- 172.21.15.191:0/1849952824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608aab60 con 0x55a960cd5400 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c55740700 1 --2- 172.21.15.191:0/1849952824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55a960c775f0 tx=0x55a960bd1f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e0e66a1eeb2fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c3c70e700 1 -- 172.21.15.191:0/1849952824 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a960cf2000 con 0x55a960cd5400 2024-04-27T04:34:45.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c3c70e700 1 -- 172.21.15.191:0/1849952824 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960cfc000 con 0x55a960cd5400 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.416+0000 7f7c3c70e700 1 -- 172.21.15.191:0/1849952824 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a960cf2160 con 0x55a960cd5400 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- 172.21.15.191:0/1849952824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 msgr2=0x55a960b6c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/1849952824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55a960c775f0 tx=0x55a960bd1f80 comp rx=0 tx=0).stop 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- 172.21.15.191:0/1849952824 shutdown_connections 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/1849952824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- 172.21.15.191:0/1849952824 >> 172.21.15.191:0/1849952824 conn(0x55a960cd5000 msgr2=0x55a960970000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- 172.21.15.191:0/1849952824 shutdown_connections 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- 172.21.15.191:0/1849952824 wait complete. 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 Processor -- start 2024-04-27T04:34:45.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- start start 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c57f97a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a960924500 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c55740700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c55740700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39594/0 (socket says 172.21.15.191:39594) 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.417+0000 7f7c55740700 1 -- 172.21.15.191:0/671030781 learned_addr learned my addr 172.21.15.191:0/671030781 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c55740700 1 -- 172.21.15.191:0/671030781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9608aad00 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c55740700 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55a960cf4210 tx=0x55a960cc04e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c3af0b700 1 -- 172.21.15.191:0/671030781 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a960cf22c0 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c3af0b700 1 -- 172.21.15.191:0/671030781 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a960cfc180 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c3af0b700 1 -- 172.21.15.191:0/671030781 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a960cf2000 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a9608aa340 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.418+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9608aab60 con 0x55a960cd5400 2024-04-27T04:34:45.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.419+0000 7f7c3af0b700 1 -- 172.21.15.191:0/671030781 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a961625b80 con 0x55a960cd5400 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.419+0000 7f7c3af0b700 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960d76000 0x55a960b6d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.419+0000 7f7c3af0b700 1 -- 172.21.15.191:0/671030781 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a9608fd500 con 0x55a960cd5400 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.419+0000 7f7c56742700 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960d76000 0x55a960b6d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.419+0000 7f7c57f97a80 5 note: GC not initialized 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.419+0000 7f7c56742700 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960d76000 0x55a960b6d700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55a960c77ec0 tx=0x55a960cc0360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c38706700 20 reqs_thread_entry: start 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c57f97a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c57f97a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c36702700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c35ee0700 20 reqs_thread_entry: start 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c57f97a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:45.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c57f97a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.420+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960cd5800 con 0x55a960883800 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.421+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv10 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.421+0000 7f7c57f97a80 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.421+0000 7f7c57f97a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.421+0000 7f7c57f97a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.421+0000 7f7c57f97a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.421+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d76400 con 0x55a960883800 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v31'11 uv11 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c57f97a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c57f97a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c57f97a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:45.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94185961949440 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d76800 con 0x55a960883800 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94185960385792 notify 133143986186 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55a960c75d40 con 0x55a960883800 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c50736700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986186 cookie 94185960385792 notifier 4436 bl.length()=533 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c50736700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c50736700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c50736700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960de8000 con 0x55a960883800 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94185961949440 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== watch-notify(notify_complete (2) cookie 94185961949440 notify 133143986186 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a960d41d40 con 0x55a960883800 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.422+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d76c00 con 0x55a960c0b400 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.423+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(66 foo:foo2 [delete] v31'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.423+0000 7f7c57f97a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.423+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94185961949440 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d77000 con 0x55a960c0b400 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.424+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== watch-notify(notify (1) cookie 94185960381184 notify 133143986181 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x55a960c75ba0 con 0x55a960c0b400 2024-04-27T04:34:45.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.424+0000 7f7c4ff35700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94185960381184 notifier 4436 bl.length()=159 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.424+0000 7f7c4ff35700 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960de8800 con 0x55a960c0b400 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.424+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 notify.0 [notify cookie 94185961949440 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.424+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== watch-notify(notify_complete (2) cookie 94185961949440 notify 133143986181 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a960c741a0 con 0x55a960c0b400 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.424+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:45.464 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:45.465 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:45.466 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:45.467 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:45.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.425+0000 7f7c3cf0f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.425+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94185961947136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d77400 con 0x55a960883800 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.426+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94185961947136] v31'93 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.426+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94185960381184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d77800 con 0x55a960c0b400 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.427+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94185960381184] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.427+0000 7f7c57f97a80 20 remove_watcher() i=0 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.427+0000 7f7c57f97a80 2 removed watcher, disabling cache 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.427+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94185960382336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d77c00 con 0x55a960883800 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.428+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94185960382336] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.428+0000 7f7c57f97a80 20 remove_watcher() i=1 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.428+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94185960383488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960de8000 con 0x55a960c0bc00 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.429+0000 7f7c55f41700 1 -- 172.21.15.191:0/3529057430 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94185960383488] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb200 con 0x55a960c0bc00 2024-04-27T04:34:45.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.429+0000 7f7c57f97a80 20 remove_watcher() i=2 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.429+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94185960384640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960de8800 con 0x55a960c0b400 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.430+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94185960384640] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccb440 con 0x55a960c0b400 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.430+0000 7f7c57f97a80 20 remove_watcher() i=3 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.430+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94185960385792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960de8c00 con 0x55a960883800 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.431+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94185960385792] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960883800 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.431+0000 7f7c57f97a80 20 remove_watcher() i=4 2024-04-27T04:34:45.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.431+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94185960386944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d76400 con 0x55a960c0b000 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.432+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94185960386944] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.432+0000 7f7c57f97a80 20 remove_watcher() i=5 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.432+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94185960388096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d76800 con 0x55a960c0b000 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94185960388096] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 20 remove_watcher() i=6 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94185960389248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a960d77400 con 0x55a960c0b000 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c56742700 1 -- 172.21.15.191:0/3529057430 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94185960389248] v31'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a960ccab40 con 0x55a960c0b000 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 20 remove_watcher() i=7 2024-04-27T04:34:45.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a960c0b000 msgr2=0x55a960b6d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a960c0b000 0x55a960b6d180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a960883800 msgr2=0x55a960b6bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a960883800 0x55a960b6bb80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a960c0b400 msgr2=0x55a960b6cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a960c0b400 0x55a960b6cc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a960c0bc00 msgr2=0x55a960b6c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.433+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a960c0bc00 0x55a960b6c100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0ac00 msgr2=0x55a960b6b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0ac00 0x55a960b6b600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55a960c0fa70 tx=0x55a960cc00c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 msgr2=0x55a960b6ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/3529057430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960883400 0x55a960b6ab00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55a96170a840 tx=0x55a960bd0a80 comp rx=0 tx=0).stop 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 reap_dead start 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c55740700 1 -- 172.21.15.191:0/3529057430 reap_dead start 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 shutdown_connections 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 >> 172.21.15.191:0/3529057430 conn(0x55a960882c00 msgr2=0x55a96096f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 shutdown_connections 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/3529057430 wait complete. 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960d76000 msgr2=0x55a960b6d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960d76000 0x55a960b6d700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55a960c77ec0 tx=0x55a960cc0360 comp rx=0 tx=0).stop 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 msgr2=0x55a960b6c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55a960cf4210 tx=0x55a960cc04e0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 shutdown_connections 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960d76000 0x55a960b6d700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/671030781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960cd5400 0x55a960b6c680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 >> 172.21.15.191:0/671030781 conn(0x55a960cd5000 msgr2=0x55a960970000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 shutdown_connections 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/671030781 wait complete. 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0a000 msgr2=0x55a960b6b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.434+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0a000 0x55a960b6b080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55a96170aed0 tx=0x55a960bd0c60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 msgr2=0x55a960b6a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55a96165b5f0 tx=0x55a960973920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 shutdown_connections 2024-04-27T04:34:45.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a960c0a000 0x55a960b6b080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 --2- 172.21.15.191:0/4032190451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a960882800 0x55a960b6a580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 >> 172.21.15.191:0/4032190451 conn(0x55a960882400 msgr2=0x55a96096f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 shutdown_connections 2024-04-27T04:34:45.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.435+0000 7f7c57f97a80 1 -- 172.21.15.191:0/4032190451 wait complete. 2024-04-27T04:34:45.480 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:45.481 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-27T04:34:45.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-27T04:34:45.481 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-27T04:34:45.526 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.527 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2efd35a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564f22e88800 0x564f23170580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2efd35a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564f231b3500 con 0x564f22e88c00 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2efd35a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x564f22f2aa00 con 0x564f22e88800 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2edcdf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564f22e88800 0x564f23170580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2ed4de700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x564f22e88c00 0x564f22f74800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55444/0 (socket says 172.21.15.191:55444) 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1209934593 learned_addr learned my addr 172.21.15.191:0/1209934593 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1209934593 >> v1:172.21.15.138:6789/0 conn(0x564f22e88c00 legacy=0x564f22f74800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1209934593 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb1d40 con 0x564f22e88800 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.525+0000 7fb2eccdd700 1 -- 172.21.15.191:0/1209934593 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266780237 0 0) 0x564f23c56000 con 0x564f22e88c00 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1209934593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x564f2316b9e0 tx=0x564f22f79920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5fc7cf2cf0b747 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2eccdd700 1 -- 172.21.15.191:0/1209934593 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f414a0 con 0x564f22e88800 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2eccdd700 1 -- 172.21.15.191:0/1209934593 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f231b3800 con 0x564f22e88800 2024-04-27T04:34:45.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2eccdd700 1 -- 172.21.15.191:0/1209934593 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f41600 con 0x564f22e88800 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1209934593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 msgr2=0x564f23170580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1209934593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x564f2316b9e0 tx=0x564f22f79920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1209934593 shutdown_connections 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1209934593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1209934593 >> 172.21.15.191:0/1209934593 conn(0x564f22e88400 msgr2=0x564f22f74000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1209934593 shutdown_connections 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.526+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1209934593 wait complete. 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2efd35a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2efd35a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f22f2a500 con 0x564f22e88800 2024-04-27T04:34:45.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2edcdf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2edcdf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39600/0 (socket says 172.21.15.191:39600) 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2edcdf700 1 -- 172.21.15.191:0/3151369445 learned_addr learned my addr 172.21.15.191:0/3151369445 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2edcdf700 1 -- 172.21.15.191:0/3151369445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb1ba0 con 0x564f22e88800 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.527+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/3151369445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x564f23c59800 tx=0x564f22f79800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ec7f1bd16bdf76 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2ec4dc700 1 -- 172.21.15.191:0/3151369445 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f41760 con 0x564f22e88800 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2ec4dc700 1 -- 172.21.15.191:0/3151369445 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f231b3b00 con 0x564f22e88800 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2ec4dc700 1 -- 172.21.15.191:0/3151369445 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f414a0 con 0x564f22e88800 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3151369445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 msgr2=0x564f23170580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/3151369445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x564f23c59800 tx=0x564f22f79800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3151369445 shutdown_connections 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/3151369445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3151369445 >> 172.21.15.191:0/3151369445 conn(0x564f22e88400 msgr2=0x564f22f75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3151369445 shutdown_connections 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3151369445 wait complete. 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.528+0000 7fb2efd35a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f22f2a3c0 con 0x564f22e88800 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2edcdf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2edcdf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39616/0 (socket says 172.21.15.191:39616) 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1193079187 learned_addr learned my addr 172.21.15.191:0/1193079187 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1193079187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb1a00 con 0x564f22e88800 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x564f23c59ec0 tx=0x564f22f79920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2eacd9700 1 -- 172.21.15.191:0/1193079187 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f41600 con 0x564f22e88800 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2eacd9700 1 -- 172.21.15.191:0/1193079187 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f231b3c80 con 0x564f22e88800 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2eacd9700 1 -- 172.21.15.191:0/1193079187 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f418c0 con 0x564f22e88800 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f22eb1860 con 0x564f22e88800 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.529+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f22eb16c0 con 0x564f22e88800 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2efd35a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2efd35a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f22f2a000 con 0x564f22e89400 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2eacd9700 1 -- 172.21.15.191:0/1193079187 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564f23c2ab00 con 0x564f22e88800 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2eacd9700 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212000 0x564f23171080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2ee4e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2eacd9700 1 -- 172.21.15.191:0/1193079187 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564f22f03180 con 0x564f22e88800 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2ee4e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39618/0 (socket says 172.21.15.191:39618) 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/4216995110 learned_addr learned my addr 172.21.15.191:0/4216995110 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212000 0x564f23171080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/4216995110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb0000 con 0x564f22e89400 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/4216995110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x564f23c599b0 tx=0x564f231d8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67786b4d07b91da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.530+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212000 0x564f23171080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564f23d03050 tx=0x564f231d8c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2e84d4700 1 -- 172.21.15.191:0/4216995110 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f41600 con 0x564f22e89400 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2e84d4700 1 -- 172.21.15.191:0/4216995110 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f22f29c80 con 0x564f22e89400 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2e84d4700 1 -- 172.21.15.191:0/4216995110 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f418c0 con 0x564f22e89400 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- 172.21.15.191:0/4216995110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 msgr2=0x564f23170b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/4216995110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x564f23c599b0 tx=0x564f231d8b40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- 172.21.15.191:0/4216995110 shutdown_connections 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/4216995110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- 172.21.15.191:0/4216995110 >> 172.21.15.191:0/4216995110 conn(0x564f22e88c00 msgr2=0x564f22f75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- 172.21.15.191:0/4216995110 shutdown_connections 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- 172.21.15.191:0/4216995110 wait complete. 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.531+0000 7fb2efd35a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f22ec4640 con 0x564f22e89400 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2ee4e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2ee4e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39624/0 (socket says 172.21.15.191:39624) 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 learned_addr learned my addr 172.21.15.191:0/1496247277 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb0ea0 con 0x564f22e89400 2024-04-27T04:34:45.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x564f23d02240 tx=0x564f22f798c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2e6cd1700 1 -- 172.21.15.191:0/1496247277 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f41760 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2e6cd1700 1 -- 172.21.15.191:0/1496247277 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f22f29b00 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f22eb0820 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2e6cd1700 1 -- 172.21.15.191:0/1496247277 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f22f41600 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.532+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f22eb09c0 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2e6cd1700 1 -- 172.21.15.191:0/1496247277 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564f23c2adc0 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2e6cd1700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212c00 0x564f23171600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2e6cd1700 1 -- 172.21.15.191:0/1496247277 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564f22f03340 con 0x564f22e89400 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212c00 0x564f23171600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564f22e89800 0x564f23171b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f22e89c00 con 0x564f22e89800 2024-04-27T04:34:45.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.533+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564f22e89800 0x564f23171b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212c00 0x564f23171600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x564f23c58030 tx=0x564f232bc0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564f22e89800 0x564f23171b80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 realm 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213000 con 0x564f22e89800 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213400 con 0x564f22e89800 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213800 con 0x564f22e89800 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.534+0000 7fb2efd35a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564f23213c00 0x564f23172100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213000 con 0x564f23213c00 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564f23213c00 0x564f23172100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564f23213c00 0x564f23172100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2efd35a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2efd35a80 20 searching for the correct realm 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.535+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213400 con 0x564f23213c00 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2edcdf700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564f23213400 0x564f23172c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213000 con 0x564f23213400 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564f23213400 0x564f23172c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564f23213400 0x564f23172c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564f23213000 0x564f23173180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.536+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564f23213000 0x564f23173180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564f23213000 0x564f23173180 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.537+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213400 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.538+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213400 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.539+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.540+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.540+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.540+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.540+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.540+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.540+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.541+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213400 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.542+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213400 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213c00 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.543+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213400 2024-04-27T04:34:45.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f22e89c00 con 0x564f22e89800 2024-04-27T04:34:45.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.544+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da400 con 0x564f22e89800 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da800 con 0x564f23213c00 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 zone default found 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 4 Realm: () 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 zonegroup default 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232dac00 con 0x564f23213000 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db000 con 0x564f23213c00 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.545+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db400 con 0x564f23213c00 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.546+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.546+0000 7fb2efd35a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.546+0000 7fb2efd35a80 20 started sync module instance, tier type = 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.546+0000 7fb2efd35a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.546+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db800 con 0x564f23213400 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.547+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.547+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94897891080960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da800 con 0x564f23213400 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.547+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db000 con 0x564f22e89800 2024-04-27T04:34:45.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.548+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94897891080960] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.548+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.548+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94897891082112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db400 con 0x564f22e89800 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.548+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da400 con 0x564f23213c00 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.549+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.549+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94897891083264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232dac00 con 0x564f23213c00 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.549+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f22e89c00 con 0x564f23213400 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.549+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94897891082112] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.550+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94897891083264] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.550+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.550+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94897891084416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f23213800 con 0x564f23213400 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.550+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f22e89800 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.551+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94897891084416] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.551+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.551+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94897891085568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db800 con 0x564f22e89800 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.551+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da800 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.552+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94897891085568] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.552+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.552+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94897891086720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232dbc00 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.552+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db000 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.553+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.553+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94897891087872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db400 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.553+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da000 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.553+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94897891086720] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94897891087872] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94897891089024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db800 con 0x564f23213000 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=0 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=1 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=2 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=3 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=4 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=5 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.554+0000 7fb2efd35a80 20 add_watcher() i=6 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.555+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94897891089024] v31'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.555+0000 7fb2efd35a80 20 add_watcher() i=7 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.555+0000 7fb2efd35a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.555+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da800 con 0x564f22e89800 2024-04-27T04:34:45.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.555+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.555+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94897892646912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232dbc00 con 0x564f22e89800 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94897892646912] v31'94 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2d4cad700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2efd35a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2efd35a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2efd35a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.556+0000 7fb2efd35a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f22ec4780 con 0x564f232db400 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2ed4de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2ed4de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39628/0 (socket says 172.21.15.191:39628) 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1142007925 learned_addr learned my addr 172.21.15.191:0/1142007925 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1142007925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb0b60 con 0x564f232db400 2024-04-27T04:34:45.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/1142007925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x564f23261800 tx=0x564f231d9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949742ffdb04c1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2d44ac700 1 -- 172.21.15.191:0/1142007925 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f232f8000 con 0x564f232db400 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2d44ac700 1 -- 172.21.15.191:0/1142007925 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f23302000 con 0x564f232db400 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.557+0000 7fb2d44ac700 1 -- 172.21.15.191:0/1142007925 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f232f8160 con 0x564f232db400 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1142007925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 msgr2=0x564f23172680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1142007925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x564f23261800 tx=0x564f231d9f80 comp rx=0 tx=0).stop 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1142007925 shutdown_connections 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1142007925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1142007925 >> 172.21.15.191:0/1142007925 conn(0x564f232db000 msgr2=0x564f22f76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1142007925 shutdown_connections 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1142007925 wait complete. 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 Processor -- start 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- start start 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.558+0000 7fb2efd35a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f22f2a000 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2ed4de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2ed4de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39632/0 (socket says 172.21.15.191:39632) 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2ed4de700 1 -- 172.21.15.191:0/3832885746 learned_addr learned my addr 172.21.15.191:0/3832885746 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2ed4de700 1 -- 172.21.15.191:0/3832885746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f22eb0d00 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2ed4de700 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x564f232fb200 tx=0x564f232bc4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2d2ca9700 1 -- 172.21.15.191:0/3832885746 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f232f82c0 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2d2ca9700 1 -- 172.21.15.191:0/3832885746 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f23302180 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f22eb0340 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2d2ca9700 1 -- 172.21.15.191:0/3832885746 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f232f8000 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.559+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f22eb0b60 con 0x564f232db400 2024-04-27T04:34:45.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.560+0000 7fb2d2ca9700 1 -- 172.21.15.191:0/3832885746 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564f23c2bb80 con 0x564f232db400 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.560+0000 7fb2d2ca9700 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f2337c000 0x564f23173700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.560+0000 7fb2d2ca9700 1 -- 172.21.15.191:0/3832885746 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564f22f03500 con 0x564f232db400 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.560+0000 7fb2efd35a80 5 note: GC not initialized 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.560+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f2337c000 0x564f23173700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.560+0000 7fb2ee4e0700 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f2337c000 0x564f23173700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564f23261470 tx=0x564f232bc360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2d04a4700 20 reqs_thread_entry: start 2024-04-27T04:34:45.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2efd35a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2efd35a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2ce4a0700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2cdc7e700 20 reqs_thread_entry: start 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2efd35a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.561+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232db800 con 0x564f22e89800 2024-04-27T04:34:45.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.562+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.562+0000 7fb2efd35a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.562+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f232da800 con 0x564f23213400 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.562+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.563+0000 7fb2d4cad700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.563+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94897892646912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337c400 con 0x564f22e89800 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.564+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94897892646912] v31'95 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.564+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94897891080960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337c800 con 0x564f23213400 2024-04-27T04:34:45.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.565+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94897891080960] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.565+0000 7fb2efd35a80 20 remove_watcher() i=0 2024-04-27T04:34:45.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.565+0000 7fb2efd35a80 2 removed watcher, disabling cache 2024-04-27T04:34:45.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.565+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94897891082112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337cc00 con 0x564f22e89800 2024-04-27T04:34:45.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.566+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94897891082112] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.566+0000 7fb2efd35a80 20 remove_watcher() i=1 2024-04-27T04:34:45.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.566+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94897891083264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337d000 con 0x564f23213c00 2024-04-27T04:34:45.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.567+0000 7fb2edcdf700 1 -- 172.21.15.191:0/1496247277 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94897891083264] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad200 con 0x564f23213c00 2024-04-27T04:34:45.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.567+0000 7fb2efd35a80 20 remove_watcher() i=2 2024-04-27T04:34:45.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.567+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94897891084416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337d400 con 0x564f23213400 2024-04-27T04:34:45.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.568+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94897891084416] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232ad440 con 0x564f23213400 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.568+0000 7fb2efd35a80 20 remove_watcher() i=3 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.568+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94897891085568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337d800 con 0x564f22e89800 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.569+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94897891085568] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f22e89800 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.569+0000 7fb2efd35a80 20 remove_watcher() i=4 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.569+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94897891086720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337dc00 con 0x564f23213000 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.570+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94897891086720] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.570+0000 7fb2efd35a80 20 remove_watcher() i=5 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.570+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94897891087872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337c800 con 0x564f23213000 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.570+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94897891087872] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.570+0000 7fb2efd35a80 20 remove_watcher() i=6 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.570+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94897891089024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f2337d400 con 0x564f23213000 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2ee4e0700 1 -- 172.21.15.191:0/1496247277 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94897891089024] v31'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f232acb40 con 0x564f23213000 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 20 remove_watcher() i=7 2024-04-27T04:34:45.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564f23213000 msgr2=0x564f23173180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564f23213000 0x564f23173180 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564f22e89800 msgr2=0x564f23171b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564f22e89800 0x564f23171b80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564f23213400 msgr2=0x564f23172c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564f23213400 0x564f23172c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564f23213c00 msgr2=0x564f23172100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564f23213c00 0x564f23172100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212c00 msgr2=0x564f23171600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212c00 0x564f23171600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x564f23c58030 tx=0x564f232bc0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 msgr2=0x564f23170b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1496247277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e89400 0x564f23170b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x564f23d02240 tx=0x564f22f798c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 reap_dead start 2024-04-27T04:34:45.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2ed4de700 1 -- 172.21.15.191:0/1496247277 reap_dead start 2024-04-27T04:34:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 shutdown_connections 2024-04-27T04:34:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.571+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 >> 172.21.15.191:0/1496247277 conn(0x564f22e88c00 msgr2=0x564f22f75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 shutdown_connections 2024-04-27T04:34:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1496247277 wait complete. 2024-04-27T04:34:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f2337c000 msgr2=0x564f23173700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f2337c000 0x564f23173700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564f23261470 tx=0x564f232bc360 comp rx=0 tx=0).stop 2024-04-27T04:34:45.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 msgr2=0x564f23172680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x564f232fb200 tx=0x564f232bc4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 shutdown_connections 2024-04-27T04:34:45.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f2337c000 0x564f23173700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/3832885746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f232db400 0x564f23172680 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 >> 172.21.15.191:0/3832885746 conn(0x564f232db000 msgr2=0x564f22f76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 shutdown_connections 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/3832885746 wait complete. 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212000 msgr2=0x564f23171080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212000 0x564f23171080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564f23d03050 tx=0x564f231d8c60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 msgr2=0x564f23170580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x564f23c59ec0 tx=0x564f22f79920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 shutdown_connections 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564f23212000 0x564f23171080 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 --2- 172.21.15.191:0/1193079187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f22e88800 0x564f23170580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.572+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 >> 172.21.15.191:0/1193079187 conn(0x564f22e88400 msgr2=0x564f22f75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.573+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 shutdown_connections 2024-04-27T04:34:45.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.573+0000 7fb2efd35a80 1 -- 172.21.15.191:0/1193079187 wait complete. 2024-04-27T04:34:45.603 INFO:teuthology.orchestra.run.smithi191.stdout:[] 2024-04-27T04:34:45.604 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T04:34:45.604 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-27T04:34:45.604 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-27T04:34:45.604 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-27T04:34:45.649 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.649 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.647+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab89afca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab89afca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555ddba8f500 con 0x555ddb764c00 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab89afca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x555ddb806a00 con 0x555ddb764800 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab87aa6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab872a5700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x555ddb764c00 0x555ddb850800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55456/0 (socket says 172.21.15.191:55456) 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab872a5700 1 -- 172.21.15.191:0/3973159187 learned_addr learned my addr 172.21.15.191:0/3973159187 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab87aa6700 1 -- 172.21.15.191:0/3973159187 >> v1:172.21.15.138:6789/0 conn(0x555ddb764c00 legacy=0x555ddb850800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab87aa6700 1 -- 172.21.15.191:0/3973159187 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78dd40 con 0x555ddb764800 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab87aa6700 1 --2- 172.21.15.191:0/3973159187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x555ddba479e0 tx=0x555ddb855920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf7d6bbe889b877 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.648+0000 7fab86aa4700 1 -- 172.21.15.191:0/3973159187 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d4a0 con 0x555ddb764800 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab86aa4700 1 -- 172.21.15.191:0/3973159187 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddba8f800 con 0x555ddb764800 2024-04-27T04:34:45.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab86aa4700 1 -- 172.21.15.191:0/3973159187 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d600 con 0x555ddb764800 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 -- 172.21.15.191:0/3973159187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 msgr2=0x555ddba4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 --2- 172.21.15.191:0/3973159187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x555ddba479e0 tx=0x555ddb855920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 -- 172.21.15.191:0/3973159187 shutdown_connections 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 --2- 172.21.15.191:0/3973159187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 -- 172.21.15.191:0/3973159187 >> 172.21.15.191:0/3973159187 conn(0x555ddb764400 msgr2=0x555ddb850000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 -- 172.21.15.191:0/3973159187 shutdown_connections 2024-04-27T04:34:45.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.649+0000 7fab89afca80 1 -- 172.21.15.191:0/3973159187 wait complete. 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab89afca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab89afca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ddb806500 con 0x555ddb764800 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab87aa6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab87aa6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39644/0 (socket says 172.21.15.191:39644) 2024-04-27T04:34:45.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab87aa6700 1 -- 172.21.15.191:0/673521161 learned_addr learned my addr 172.21.15.191:0/673521161 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab87aa6700 1 -- 172.21.15.191:0/673521161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78dba0 con 0x555ddb764800 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab87aa6700 1 --2- 172.21.15.191:0/673521161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x555ddc53d740 tx=0x555ddb855800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb301f52bbf45ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab862a3700 1 -- 172.21.15.191:0/673521161 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d760 con 0x555ddb764800 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab862a3700 1 -- 172.21.15.191:0/673521161 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddba8fb00 con 0x555ddb764800 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.650+0000 7fab862a3700 1 -- 172.21.15.191:0/673521161 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d4a0 con 0x555ddb764800 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- 172.21.15.191:0/673521161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 msgr2=0x555ddba4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 --2- 172.21.15.191:0/673521161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x555ddc53d740 tx=0x555ddb855800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- 172.21.15.191:0/673521161 shutdown_connections 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 --2- 172.21.15.191:0/673521161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- 172.21.15.191:0/673521161 >> 172.21.15.191:0/673521161 conn(0x555ddb764400 msgr2=0x555ddb851000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- 172.21.15.191:0/673521161 shutdown_connections 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- 172.21.15.191:0/673521161 wait complete. 2024-04-27T04:34:45.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.651+0000 7fab89afca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ddb8063c0 con 0x555ddb764800 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab87aa6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab87aa6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39656/0 (socket says 172.21.15.191:39656) 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab87aa6700 1 -- 172.21.15.191:0/2853889631 learned_addr learned my addr 172.21.15.191:0/2853889631 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab87aa6700 1 -- 172.21.15.191:0/2853889631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78da00 con 0x555ddb764800 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab87aa6700 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x555ddc53d5f0 tx=0x555ddb855920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab84aa0700 1 -- 172.21.15.191:0/2853889631 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d600 con 0x555ddb764800 2024-04-27T04:34:45.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab84aa0700 1 -- 172.21.15.191:0/2853889631 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddba8fc80 con 0x555ddb764800 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab84aa0700 1 -- 172.21.15.191:0/2853889631 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d8c0 con 0x555ddb764800 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555ddb78d860 con 0x555ddb764800 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.652+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ddb78d6c0 con 0x555ddb764800 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab84aa0700 1 -- 172.21.15.191:0/2853889631 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555ddc506b00 con 0x555ddb764800 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab89afca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab89afca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ddb806000 con 0x555ddb765400 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab84aa0700 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaee000 0x555ddba4d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab84aa0700 1 -- 172.21.15.191:0/2853889631 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x555ddb7df180 con 0x555ddb764800 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab882a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab882a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39660/0 (socket says 172.21.15.191:39660) 2024-04-27T04:34:45.676 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab882a7700 1 -- 172.21.15.191:0/2305443149 learned_addr learned my addr 172.21.15.191:0/2305443149 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab87aa6700 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaee000 0x555ddba4d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab882a7700 1 -- 172.21.15.191:0/2305443149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78c000 con 0x555ddb765400 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab87aa6700 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaee000 0x555ddba4d080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x555ddc5f0ed0 tx=0x555ddbab2c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.653+0000 7fab882a7700 1 --2- 172.21.15.191:0/2305443149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x555ddc5f0a50 tx=0x555ddbab2b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5800d5a5ddca78f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab8229b700 1 -- 172.21.15.191:0/2305443149 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d600 con 0x555ddb765400 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab8229b700 1 -- 172.21.15.191:0/2305443149 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddb805c80 con 0x555ddb765400 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab8229b700 1 -- 172.21.15.191:0/2305443149 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d8c0 con 0x555ddb765400 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 -- 172.21.15.191:0/2305443149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 msgr2=0x555ddba4cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 --2- 172.21.15.191:0/2305443149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x555ddc5f0a50 tx=0x555ddbab2b40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 -- 172.21.15.191:0/2305443149 shutdown_connections 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 --2- 172.21.15.191:0/2305443149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 -- 172.21.15.191:0/2305443149 >> 172.21.15.191:0/2305443149 conn(0x555ddb764c00 msgr2=0x555ddb851800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 -- 172.21.15.191:0/2305443149 shutdown_connections 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 -- 172.21.15.191:0/2305443149 wait complete. 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.654+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab89afca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab89afca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ddb7a0640 con 0x555ddb765400 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab882a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab882a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39662/0 (socket says 172.21.15.191:39662) 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 learned_addr learned my addr 172.21.15.191:0/3965164870 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78cea0 con 0x555ddb765400 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab882a7700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x555ddc5f0840 tx=0x555ddb855ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab80a98700 1 -- 172.21.15.191:0/3965164870 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d760 con 0x555ddb765400 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab80a98700 1 -- 172.21.15.191:0/3965164870 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddb805b00 con 0x555ddb765400 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555ddb78c820 con 0x555ddb765400 2024-04-27T04:34:45.678 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab80a98700 1 -- 172.21.15.191:0/3965164870 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddb81d600 con 0x555ddb765400 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.655+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ddb78c9c0 con 0x555ddb765400 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab80a98700 1 -- 172.21.15.191:0/3965164870 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555ddc506dc0 con 0x555ddb765400 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab80a98700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaeec00 0x555ddba4d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab80a98700 1 -- 172.21.15.191:0/3965164870 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x555ddb7df340 con 0x555ddb765400 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab872a5700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaeec00 0x555ddba4d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555ddb765800 0x555ddba4db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.656+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddb765c00 con 0x555ddb765800 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab882a7700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555ddb765800 0x555ddba4db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab872a5700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaeec00 0x555ddba4d600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x555ddbaf1c80 tx=0x555ddbb9a0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab882a7700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555ddb765800 0x555ddba4db80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 20 realm 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef000 con 0x555ddb765800 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.657+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef400 con 0x555ddb765800 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef800 con 0x555ddb765800 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555ddbaefc00 0x555ddba4e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef000 con 0x555ddbaefc00 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab87aa6700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555ddbaefc00 0x555ddba4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab87aa6700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555ddbaefc00 0x555ddba4e100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.658+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab89afca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab89afca80 20 searching for the correct realm 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef400 con 0x555ddbaefc00 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.681 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab87aa6700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555ddbaef400 0x555ddba4ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef000 con 0x555ddbaef400 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab872a5700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555ddbaef400 0x555ddba4ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.659+0000 7fab872a5700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555ddbaef400 0x555ddba4ec00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab872a5700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555ddbaef000 0x555ddba4f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab882a7700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555ddbaef000 0x555ddba4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab882a7700 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555ddbaef000 0x555ddba4f180 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.660+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.661+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef400 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.662+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef400 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.663+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.664+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef400 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.665+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.666+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.666+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.666+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.666+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.667+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.667+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef400 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.667+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.667+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaefc00 2024-04-27T04:34:45.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.667+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.667+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef400 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddb765c00 con 0x555ddb765800 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.668+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8400 con 0x555ddb765800 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8800 con 0x555ddbaefc00 2024-04-27T04:34:45.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 zone default found 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 4 Realm: () 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 zonegroup default 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8c00 con 0x555ddbaef000 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9000 con 0x555ddbaefc00 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9400 con 0x555ddbaefc00 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.669+0000 7fab89afca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.670+0000 7fab89afca80 20 started sync module instance, tier type = 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.670+0000 7fab89afca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.670+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9800 con 0x555ddbaef400 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.671+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.671+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93861605292800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8800 con 0x555ddbaef400 2024-04-27T04:34:45.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.671+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9000 con 0x555ddb765800 2024-04-27T04:34:45.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.672+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93861605292800] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.672+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.672+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93861605293952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9400 con 0x555ddb765800 2024-04-27T04:34:45.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.672+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8400 con 0x555ddbaefc00 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.673+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.673+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93861605295104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8c00 con 0x555ddbaefc00 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.673+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddb765c00 con 0x555ddbaef400 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.673+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93861605293952] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.674+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93861605295104] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.674+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.674+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93861605296256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbaef800 con 0x555ddbaef400 2024-04-27T04:34:45.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.674+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddb765800 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.675+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93861605296256] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.675+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.675+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93861605297408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9800 con 0x555ddb765800 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.675+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8800 con 0x555ddbaef000 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.676+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93861605297408] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.676+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.676+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93861605298560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9c00 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.676+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9000 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.677+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93861605298560] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.677+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.677+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93861605299712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9400 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.677+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8000 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93861605299712] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93861605300864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9800 con 0x555ddbaef000 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=0 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=1 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=2 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=3 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=4 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=5 2024-04-27T04:34:45.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.678+0000 7fab89afca80 20 add_watcher() i=6 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.679+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93861605300864] v31'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.679+0000 7fab89afca80 20 add_watcher() i=7 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.679+0000 7fab89afca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.679+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8800 con 0x555ddb765800 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.679+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.679+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93861606866944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9c00 con 0x555ddb765800 2024-04-27T04:34:45.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.680+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93861606866944] v31'96 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab6ea74700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab89afca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab89afca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab89afca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab89afca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ddb7a0780 con 0x555ddbbb9400 2024-04-27T04:34:45.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab872a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab872a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39676/0 (socket says 172.21.15.191:39676) 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab872a5700 1 -- 172.21.15.191:0/3953771875 learned_addr learned my addr 172.21.15.191:0/3953771875 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.681+0000 7fab872a5700 1 -- 172.21.15.191:0/3953771875 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78cb60 con 0x555ddbbb9400 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab872a5700 1 --2- 172.21.15.191:0/3953771875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x555ddbb495f0 tx=0x555ddbab3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c36d4a796b1a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab6e273700 1 -- 172.21.15.191:0/3953771875 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddbbd6000 con 0x555ddbbb9400 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab6e273700 1 -- 172.21.15.191:0/3953771875 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddbbe0000 con 0x555ddbbb9400 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab6e273700 1 -- 172.21.15.191:0/3953771875 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddbbd6160 con 0x555ddbbb9400 2024-04-27T04:34:45.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 -- 172.21.15.191:0/3953771875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 msgr2=0x555ddba4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 --2- 172.21.15.191:0/3953771875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x555ddbb495f0 tx=0x555ddbab3f80 comp rx=0 tx=0).stop 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 -- 172.21.15.191:0/3953771875 shutdown_connections 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 --2- 172.21.15.191:0/3953771875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 -- 172.21.15.191:0/3953771875 >> 172.21.15.191:0/3953771875 conn(0x555ddbbb9000 msgr2=0x555ddb852000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 -- 172.21.15.191:0/3953771875 shutdown_connections 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.682+0000 7fab89afca80 1 -- 172.21.15.191:0/3953771875 wait complete. 2024-04-27T04:34:45.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab89afca80 1 Processor -- start 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab89afca80 1 -- start start 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab89afca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab89afca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ddb806000 con 0x555ddbbb9400 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab872a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab872a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39678/0 (socket says 172.21.15.191:39678) 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.683+0000 7fab872a5700 1 -- 172.21.15.191:0/2541282964 learned_addr learned my addr 172.21.15.191:0/2541282964 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab872a5700 1 -- 172.21.15.191:0/2541282964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ddb78cd00 con 0x555ddbbb9400 2024-04-27T04:34:45.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab872a5700 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x555ddbbd8210 tx=0x555ddbb9a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab6ca70700 1 -- 172.21.15.191:0/2541282964 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddbbd62c0 con 0x555ddbbb9400 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab6ca70700 1 -- 172.21.15.191:0/2541282964 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ddbbe0180 con 0x555ddbbb9400 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab6ca70700 1 -- 172.21.15.191:0/2541282964 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555ddbbd6000 con 0x555ddbbb9400 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555ddb78c340 con 0x555ddbbb9400 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.684+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ddb78cb60 con 0x555ddbbb9400 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.685+0000 7fab6ca70700 1 -- 172.21.15.191:0/2541282964 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555ddc507b80 con 0x555ddbbb9400 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.685+0000 7fab6ca70700 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbc58000 0x555ddba4f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.685+0000 7fab6ca70700 1 -- 172.21.15.191:0/2541282964 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x555ddb7df500 con 0x555ddbbb9400 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.685+0000 7fab882a7700 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbc58000 0x555ddba4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.685+0000 7fab89afca80 5 note: GC not initialized 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.685+0000 7fab882a7700 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbc58000 0x555ddba4f700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x555ddbb49ec0 tx=0x555ddbb9a360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab6a26b700 20 reqs_thread_entry: start 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab89afca80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab89afca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab68267700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:15 2024 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab67a45700 20 reqs_thread_entry: start 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab89afca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb9800 con 0x555ddb765800 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.686+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.687+0000 7fab89afca80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.687+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbbb8800 con 0x555ddbaef400 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.687+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.687+0000 7fab6ea74700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.687+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93861606866944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc58400 con 0x555ddb765800 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.688+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93861606866944] v31'97 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.688+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93861605292800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc58800 con 0x555ddbaef400 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.689+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93861605292800] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.689+0000 7fab89afca80 20 remove_watcher() i=0 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.689+0000 7fab89afca80 2 removed watcher, disabling cache 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.689+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93861605293952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc58c00 con 0x555ddb765800 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.690+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93861605293952] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.690+0000 7fab89afca80 20 remove_watcher() i=1 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.690+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93861605295104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc59000 con 0x555ddbaefc00 2024-04-27T04:34:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.691+0000 7fab87aa6700 1 -- 172.21.15.191:0/3965164870 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93861605295104] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91200 con 0x555ddbaefc00 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.691+0000 7fab89afca80 20 remove_watcher() i=2 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.691+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93861605296256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc59400 con 0x555ddbaef400 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.692+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93861605296256] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb91440 con 0x555ddbaef400 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.692+0000 7fab89afca80 20 remove_watcher() i=3 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.692+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93861605297408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc59800 con 0x555ddb765800 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.693+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93861605297408] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddb765800 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.693+0000 7fab89afca80 20 remove_watcher() i=4 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.693+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93861605298560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc59c00 con 0x555ddbaef000 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.694+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93861605298560] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.694+0000 7fab89afca80 20 remove_watcher() i=5 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.694+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93861605299712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc58800 con 0x555ddbaef000 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.695+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93861605299712] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.695+0000 7fab89afca80 20 remove_watcher() i=6 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.695+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93861605300864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x555ddbc59400 con 0x555ddbaef000 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab882a7700 1 -- 172.21.15.191:0/3965164870 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93861605300864] v31'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555ddbb90b40 con 0x555ddbaef000 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 20 remove_watcher() i=7 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555ddbaef000 msgr2=0x555ddba4f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555ddbaef000 0x555ddba4f180 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555ddb765800 msgr2=0x555ddba4db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555ddb765800 0x555ddba4db80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555ddbaef400 msgr2=0x555ddba4ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555ddbaef400 0x555ddba4ec00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555ddbaefc00 msgr2=0x555ddba4e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555ddbaefc00 0x555ddba4e100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaeec00 msgr2=0x555ddba4d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaeec00 0x555ddba4d600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x555ddbaf1c80 tx=0x555ddbb9a0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 msgr2=0x555ddba4cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 --2- 172.21.15.191:0/3965164870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb765400 0x555ddba4cb00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x555ddc5f0840 tx=0x555ddb855ce0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 reap_dead start 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab872a5700 1 -- 172.21.15.191:0/3965164870 reap_dead start 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 shutdown_connections 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 >> 172.21.15.191:0/3965164870 conn(0x555ddb764c00 msgr2=0x555ddb851800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 shutdown_connections 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.696+0000 7fab89afca80 1 -- 172.21.15.191:0/3965164870 wait complete. 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbc58000 msgr2=0x555ddba4f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbc58000 0x555ddba4f700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x555ddbb49ec0 tx=0x555ddbb9a360 comp rx=0 tx=0).stop 2024-04-27T04:34:45.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 msgr2=0x555ddba4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x555ddbbd8210 tx=0x555ddbb9a4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 shutdown_connections 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbc58000 0x555ddba4f700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2541282964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddbbb9400 0x555ddba4e680 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 >> 172.21.15.191:0/2541282964 conn(0x555ddbbb9000 msgr2=0x555ddb852000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 shutdown_connections 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2541282964 wait complete. 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaee000 msgr2=0x555ddba4d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaee000 0x555ddba4d080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x555ddc5f0ed0 tx=0x555ddbab2c60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 msgr2=0x555ddba4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x555ddc53d5f0 tx=0x555ddb855920 comp rx=0 tx=0).stop 2024-04-27T04:34:45.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 shutdown_connections 2024-04-27T04:34:45.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555ddbaee000 0x555ddba4d080 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 --2- 172.21.15.191:0/2853889631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddb764800 0x555ddba4c580 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 >> 172.21.15.191:0/2853889631 conn(0x555ddb764400 msgr2=0x555ddb851000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 shutdown_connections 2024-04-27T04:34:45.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.697+0000 7fab89afca80 1 -- 172.21.15.191:0/2853889631 wait complete. 2024-04-27T04:34:45.725 INFO:teuthology.orchestra.run.smithi191.stdout:[] 2024-04-27T04:34:45.725 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T04:34:45.741 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:45.741 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-27T04:34:45.741 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-27T04:34:45.741 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-27T04:34:45.810 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.814 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:45.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.808+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdf74aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x559045008800 0x5590452f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdf74aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559045333500 con 0x559045008c00 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdf74aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5590450aaa00 con 0x559045008800 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdcef3700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x559045008c00 0x5590450f4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55462/0 (socket says 172.21.15.191:55462) 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdcef3700 1 -- 172.21.15.191:0/3536507322 learned_addr learned my addr 172.21.15.191:0/3536507322 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/3536507322 >> v2:172.21.15.138:3300/0 conn(0x559045008800 0x5590452f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdc6f2700 1 -- 172.21.15.191:0/3536507322 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571910307 0 0) 0x559045dd6000 con 0x559045008c00 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdc6f2700 1 -- 172.21.15.191:0/3536507322 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x559045333800 con 0x559045008c00 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/3536507322 >> v1:172.21.15.138:6789/0 conn(0x559045008c00 legacy=0x5590450f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/3536507322 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045031d40 con 0x559045008800 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.809+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/3536507322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5590452eb9e0 tx=0x5590450f9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f365052f8cde9a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdc6f2700 1 -- 172.21.15.191:0/3536507322 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c14a0 con 0x559045008800 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdc6f2700 1 -- 172.21.15.191:0/3536507322 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559045333500 con 0x559045008800 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdc6f2700 1 -- 172.21.15.191:0/3536507322 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c1600 con 0x559045008800 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/3536507322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 msgr2=0x5590452f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/3536507322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5590452eb9e0 tx=0x5590450f9980 comp rx=0 tx=0).stop 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/3536507322 shutdown_connections 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/3536507322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/3536507322 >> 172.21.15.191:0/3536507322 conn(0x559045008400 msgr2=0x5590450f4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/3536507322 shutdown_connections 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.810+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/3536507322 wait complete. 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdf74aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdf74aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590450aa500 con 0x559045008800 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdd6f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdd6f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39698/0 (socket says 172.21.15.191:39698) 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/2567675046 learned_addr learned my addr 172.21.15.191:0/2567675046 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/2567675046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045031ba0 con 0x559045008800 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.811+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/2567675046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x559045dd9800 tx=0x5590450f9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de45547dba6dbe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdbef1700 1 -- 172.21.15.191:0/2567675046 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c1760 con 0x559045008800 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdbef1700 1 -- 172.21.15.191:0/2567675046 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559045333980 con 0x559045008800 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdbef1700 1 -- 172.21.15.191:0/2567675046 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c11e0 con 0x559045008800 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2567675046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 msgr2=0x5590452f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2567675046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x559045dd9800 tx=0x5590450f9800 comp rx=0 tx=0).stop 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2567675046 shutdown_connections 2024-04-27T04:34:45.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2567675046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2567675046 >> 172.21.15.191:0/2567675046 conn(0x559045008400 msgr2=0x5590450f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2567675046 shutdown_connections 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2567675046 wait complete. 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.812+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdf74aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdf74aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590450aa3c0 con 0x559045008800 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdd6f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdd6f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39714/0 (socket says 172.21.15.191:39714) 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/2438982021 learned_addr learned my addr 172.21.15.191:0/2438982021 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/2438982021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045031a00 con 0x559045008800 2024-04-27T04:34:45.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x559045dd9ec0 tx=0x5590450f9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bda6ee700 1 -- 172.21.15.191:0/2438982021 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c14a0 con 0x559045008800 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bda6ee700 1 -- 172.21.15.191:0/2438982021 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559045333b00 con 0x559045008800 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bda6ee700 1 -- 172.21.15.191:0/2438982021 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c1600 con 0x559045008800 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559045031860 con 0x559045008800 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.813+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590450316c0 con 0x559045008800 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bdf74aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bdf74aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590450aa000 con 0x559045009400 2024-04-27T04:34:45.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bda6ee700 1 -- 172.21.15.191:0/2438982021 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559045daab00 con 0x559045008800 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bddef5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bddef5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39720/0 (socket says 172.21.15.191:39720) 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bddef5700 1 -- 172.21.15.191:0/2779676323 learned_addr learned my addr 172.21.15.191:0/2779676323 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bda6ee700 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0000 0x5590452f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bda6ee700 1 -- 172.21.15.191:0/2438982021 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x559045083180 con 0x559045008800 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0000 0x5590452f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.814+0000 7f6bddef5700 1 -- 172.21.15.191:0/2779676323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045030000 con 0x559045009400 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bddef5700 1 --2- 172.21.15.191:0/2779676323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55904539c2d0 tx=0x559045360b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d82895254035675 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0000 0x5590452f1080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x559045e850e0 tx=0x559045360c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bd7ee9700 1 -- 172.21.15.191:0/2779676323 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c14a0 con 0x559045009400 2024-04-27T04:34:45.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bd7ee9700 1 -- 172.21.15.191:0/2779676323 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590450a9c80 con 0x559045009400 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bd7ee9700 1 -- 172.21.15.191:0/2779676323 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c1600 con 0x559045009400 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2779676323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 msgr2=0x5590452f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2779676323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55904539c2d0 tx=0x559045360b40 comp rx=0 tx=0).stop 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2779676323 shutdown_connections 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2779676323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2779676323 >> 172.21.15.191:0/2779676323 conn(0x559045008c00 msgr2=0x5590450f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2779676323 shutdown_connections 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2779676323 wait complete. 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.815+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bdf74aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bdf74aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559045044640 con 0x559045009400 2024-04-27T04:34:45.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bddef5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bddef5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39730/0 (socket says 172.21.15.191:39730) 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 learned_addr learned my addr 172.21.15.191:0/1665064154 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045030ea0 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bddef5700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x559045e842d0 tx=0x5590453605a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bd66e6700 1 -- 172.21.15.191:0/1665064154 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c1760 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bd66e6700 1 -- 172.21.15.191:0/1665064154 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590450a9b00 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559045030820 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bd66e6700 1 -- 172.21.15.191:0/1665064154 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590450c14a0 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.816+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590450309c0 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bd66e6700 1 -- 172.21.15.191:0/1665064154 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559045daadc0 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bd66e6700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0c00 0x5590452f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bd66e6700 1 -- 172.21.15.191:0/1665064154 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x559045083340 con 0x559045009400 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0c00 0x5590452f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559045009800 0x5590452f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559045009c00 con 0x559045009800 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.817+0000 7f6bddef5700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559045009800 0x5590452f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0c00 0x5590452f1600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x559045dd8030 tx=0x5590454480c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bddef5700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559045009800 0x5590452f1b80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 realm 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1000 con 0x559045009800 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1400 con 0x559045009800 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1800 con 0x559045009800 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.818+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5590453a1c00 0x5590452f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1000 con 0x5590453a1c00 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5590453a1c00 0x5590452f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5590453a1c00 0x5590452f2100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 20 searching for the correct realm 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.819+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1400 con 0x5590453a1c00 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdd6f4700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5590453a1400 0x5590452f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1000 con 0x5590453a1400 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5590453a1400 0x5590452f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5590453a1400 0x5590452f2c00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5590453a1000 0x5590452f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.820+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.821+0000 7f6bddef5700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5590453a1000 0x5590452f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.821+0000 7f6bddef5700 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5590453a1000 0x5590452f3180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.821+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.821+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.821+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.821+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1400 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.822+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1400 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.823+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.824+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1400 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.825+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.826+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1400 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1c00 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.827+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1400 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559045009c00 con 0x559045009800 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a400 con 0x559045009800 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a800 con 0x5590453a1c00 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 zone default found 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 4 Realm: () 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 zonegroup default 2024-04-27T04:34:45.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.828+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545ac00 con 0x5590453a1000 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b000 con 0x5590453a1c00 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b400 con 0x5590453a1c00 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 started sync module instance, tier type = 2024-04-27T04:34:45.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.829+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b800 con 0x5590453a1400 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.830+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.830+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94078124325632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a800 con 0x5590453a1400 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.830+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b000 con 0x559045009800 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.831+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94078124325632] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.831+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.831+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94078124326784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b400 con 0x559045009800 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.831+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a400 con 0x5590453a1c00 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.832+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94078124326784] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.832+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.832+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94078124327936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545ac00 con 0x5590453a1c00 2024-04-27T04:34:45.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.832+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x559045009c00 con 0x5590453a1400 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.833+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94078124327936] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.833+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.833+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94078124329088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590453a1800 con 0x5590453a1400 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.833+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x559045009800 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.834+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94078124329088] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.834+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.834+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94078124330240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b800 con 0x559045009800 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.834+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a800 con 0x5590453a1000 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.835+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.835+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94078124331392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545bc00 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.835+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b000 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.835+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94078124330240] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.836+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94078124331392] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.836+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.836+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94078124332544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b400 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.836+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a000 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94078124332544] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94078124333696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b800 con 0x5590453a1000 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=0 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=1 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=2 2024-04-27T04:34:45.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=3 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=4 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=5 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.837+0000 7f6bdf74aa80 20 add_watcher() i=6 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.838+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94078124333696] v31'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.838+0000 7f6bdf74aa80 20 add_watcher() i=7 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.838+0000 7f6bdf74aa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.838+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a800 con 0x559045009800 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.838+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.838+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94078125891584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545bc00 con 0x559045009800 2024-04-27T04:34:45.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94078125891584] v31'98 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bc46c2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdf74aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:45.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdf74aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:45.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdf74aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdf74aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559045044780 con 0x55904545b400 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdcef3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdcef3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39736/0 (socket says 172.21.15.191:39736) 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.840+0000 7f6bdcef3700 1 -- 172.21.15.191:0/530239237 learned_addr learned my addr 172.21.15.191:0/530239237 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.841+0000 7f6bdcef3700 1 -- 172.21.15.191:0/530239237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045030b60 con 0x55904545b400 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.841+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/530239237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5590453ef800 tx=0x559045361f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b7bcecf6cd3fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.841+0000 7f6bc3ec1700 1 -- 172.21.15.191:0/530239237 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559045476000 con 0x55904545b400 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.841+0000 7f6bc3ec1700 1 -- 172.21.15.191:0/530239237 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55904546e180 con 0x55904545b400 2024-04-27T04:34:45.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.841+0000 7f6bc3ec1700 1 -- 172.21.15.191:0/530239237 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559045476160 con 0x55904545b400 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/530239237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 msgr2=0x5590452f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/530239237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5590453ef800 tx=0x559045361f80 comp rx=0 tx=0).stop 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/530239237 shutdown_connections 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/530239237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/530239237 >> 172.21.15.191:0/530239237 conn(0x55904545b000 msgr2=0x5590450f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/530239237 shutdown_connections 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/530239237 wait complete. 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 Processor -- start 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- start start 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdf74aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590450aa000 con 0x55904545b400 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdcef3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdcef3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39742/0 (socket says 172.21.15.191:39742) 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.842+0000 7f6bdcef3700 1 -- 172.21.15.191:0/4156151975 learned_addr learned my addr 172.21.15.191:0/4156151975 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bdcef3700 1 -- 172.21.15.191:0/4156151975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559045030d00 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bdcef3700 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x559045479200 tx=0x5590454484e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bc26be700 1 -- 172.21.15.191:0/4156151975 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590454762c0 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bc26be700 1 -- 172.21.15.191:0/4156151975 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55904546e300 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559045030340 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bc26be700 1 -- 172.21.15.191:0/4156151975 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559045476000 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.843+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559045030b60 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.844+0000 7f6bc26be700 1 -- 172.21.15.191:0/4156151975 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559045dabb80 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.844+0000 7f6bc26be700 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590454f6000 0x5590452f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.844+0000 7f6bc26be700 1 -- 172.21.15.191:0/4156151975 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x559045083500 con 0x55904545b400 2024-04-27T04:34:45.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.844+0000 7f6bdf74aa80 5 note: GC not initialized 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.844+0000 7f6bddef5700 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590454f6000 0x5590452f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.844+0000 7f6bddef5700 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590454f6000 0x5590452f3700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5590453ef470 tx=0x559045448360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bbfeb9700 20 reqs_thread_entry: start 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bdf74aa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bdf74aa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bbdeb5700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:16 2024 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bbd693700 20 reqs_thread_entry: start 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bdf74aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545b800 con 0x559045009800 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bdf74aa80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.845+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55904545a800 con 0x5590453a1400 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.846+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.846+0000 7f6bc46c2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.846+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94078125891584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f6400 con 0x559045009800 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.847+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94078125891584] v31'99 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.847+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94078124325632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f6800 con 0x5590453a1400 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.848+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94078124325632] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.848+0000 7f6bdf74aa80 20 remove_watcher() i=0 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.848+0000 7f6bdf74aa80 2 removed watcher, disabling cache 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.848+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94078124326784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f6c00 con 0x559045009800 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.850+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94078124326784] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.850+0000 7f6bdf74aa80 20 remove_watcher() i=1 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.850+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94078124327936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f7000 con 0x5590453a1c00 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.851+0000 7f6bdd6f4700 1 -- 172.21.15.191:0/1665064154 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94078124327936] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b200 con 0x5590453a1c00 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.851+0000 7f6bdf74aa80 20 remove_watcher() i=2 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.851+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94078124329088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f7400 con 0x5590453a1400 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.852+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94078124329088] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544b440 con 0x5590453a1400 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.852+0000 7f6bdf74aa80 20 remove_watcher() i=3 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.852+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94078124330240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f7800 con 0x559045009800 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.852+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94078124330240] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x559045009800 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.852+0000 7f6bdf74aa80 20 remove_watcher() i=4 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.852+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94078124331392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f7c00 con 0x5590453a1000 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.854+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94078124331392] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.854+0000 7f6bdf74aa80 20 remove_watcher() i=5 2024-04-27T04:34:45.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.854+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94078124332544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f6800 con 0x5590453a1000 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94078124332544] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bdf74aa80 20 remove_watcher() i=6 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94078124333696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5590454f7400 con 0x5590453a1000 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bddef5700 1 -- 172.21.15.191:0/1665064154 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94078124333696] v31'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55904544ab40 con 0x5590453a1000 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bdf74aa80 20 remove_watcher() i=7 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5590453a1000 msgr2=0x5590452f3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.855+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5590453a1000 0x5590452f3180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559045009800 msgr2=0x5590452f1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x559045009800 0x5590452f1b80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5590453a1400 msgr2=0x5590452f2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5590453a1400 0x5590452f2c00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5590453a1c00 msgr2=0x5590452f2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5590453a1c00 0x5590452f2100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0c00 msgr2=0x5590452f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0c00 0x5590452f1600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x559045dd8030 tx=0x5590454480c0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 msgr2=0x5590452f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/1665064154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045009400 0x5590452f0b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x559045e842d0 tx=0x5590453605a0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 reap_dead start 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdcef3700 1 -- 172.21.15.191:0/1665064154 reap_dead start 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 shutdown_connections 2024-04-27T04:34:45.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 >> 172.21.15.191:0/1665064154 conn(0x559045008c00 msgr2=0x5590450f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 shutdown_connections 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/1665064154 wait complete. 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590454f6000 msgr2=0x5590452f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590454f6000 0x5590452f3700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5590453ef470 tx=0x559045448360 comp rx=0 tx=0).stop 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 msgr2=0x5590452f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x559045479200 tx=0x5590454484e0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 shutdown_connections 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590454f6000 0x5590452f3700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/4156151975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55904545b400 0x5590452f2680 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 >> 172.21.15.191:0/4156151975 conn(0x55904545b000 msgr2=0x5590450f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 shutdown_connections 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.856+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/4156151975 wait complete. 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0000 msgr2=0x5590452f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0000 0x5590452f1080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x559045e850e0 tx=0x559045360c60 comp rx=0 tx=0).stop 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 msgr2=0x5590452f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x559045dd9ec0 tx=0x5590450f9980 comp rx=0 tx=0).stop 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 shutdown_connections 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5590453a0000 0x5590452f1080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 --2- 172.21.15.191:0/2438982021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559045008800 0x5590452f0580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 >> 172.21.15.191:0/2438982021 conn(0x559045008400 msgr2=0x5590450f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 shutdown_connections 2024-04-27T04:34:45.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.857+0000 7f6bdf74aa80 1 -- 172.21.15.191:0/2438982021 wait complete. 2024-04-27T04:34:45.887 INFO:teuthology.orchestra.run.smithi191.stdout:[ 2024-04-27T04:34:45.887 INFO:teuthology.orchestra.run.smithi191.stdout: "myfoo" 2024-04-27T04:34:45.887 INFO:teuthology.orchestra.run.smithi191.stdout:] 2024-04-27T04:34:45.887 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-27T04:34:45.893 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:45.893 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-27T04:34:45.893 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-27T04:34:45.893 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-27T04:34:45.971 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:45.971 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.969+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.969+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.969+0000 7fc225ac3a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cbe51b2c00 0x55cbe549a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.969+0000 7fc225ac3a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cbe54dd500 con 0x55cbe51b2800 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.969+0000 7fc225ac3a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cbe5254a00 con 0x55cbe51b2c00 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc22326c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cbe51b2c00 0x55cbe549a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc223a6d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55cbe51b2800 0x55cbe529e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:55472/0 (socket says 172.21.15.191:55472) 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc223a6d700 1 -- 172.21.15.191:0/1316340795 learned_addr learned my addr 172.21.15.191:0/1316340795 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc22326c700 1 -- 172.21.15.191:0/1316340795 >> v1:172.21.15.138:6789/0 conn(0x55cbe51b2800 legacy=0x55cbe529e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc22326c700 1 -- 172.21.15.191:0/1316340795 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51dbd40 con 0x55cbe51b2c00 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc222a6b700 1 -- 172.21.15.191:0/1316340795 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054722714 0 0) 0x55cbe5f80000 con 0x55cbe51b2800 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc22326c700 1 --2- 172.21.15.191:0/1316340795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2c00 0x55cbe549a580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55cbe5495890 tx=0x55cbe52a3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ebb9e66046204c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc222a6b700 1 -- 172.21.15.191:0/1316340795 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b4a0 con 0x55cbe51b2c00 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc222a6b700 1 -- 172.21.15.191:0/1316340795 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe54dd800 con 0x55cbe51b2c00 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.970+0000 7fc222a6b700 1 -- 172.21.15.191:0/1316340795 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b600 con 0x55cbe51b2c00 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1316340795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2c00 msgr2=0x55cbe549a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/1316340795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2c00 0x55cbe549a580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55cbe5495890 tx=0x55cbe52a3920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1316340795 shutdown_connections 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/1316340795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2c00 0x55cbe549a580 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1316340795 >> 172.21.15.191:0/1316340795 conn(0x55cbe51b2400 msgr2=0x55cbe529e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1316340795 shutdown_connections 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1316340795 wait complete. 2024-04-27T04:34:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.971+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc225ac3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc225ac3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe5254500 con 0x55cbe51b2800 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39754/0 (socket says 172.21.15.191:39754) 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc223a6d700 1 -- 172.21.15.191:0/2329686925 learned_addr learned my addr 172.21.15.191:0/2329686925 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc223a6d700 1 -- 172.21.15.191:0/2329686925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51dbba0 con 0x55cbe51b2800 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc223a6d700 1 --2- 172.21.15.191:0/2329686925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55cbe5f83770 tx=0x55cbe52a3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efdb45bc8a19809 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc22226a700 1 -- 172.21.15.191:0/2329686925 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b1e0 con 0x55cbe51b2800 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc22226a700 1 -- 172.21.15.191:0/2329686925 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe54ddb00 con 0x55cbe51b2800 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.972+0000 7fc22226a700 1 -- 172.21.15.191:0/2329686925 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b4a0 con 0x55cbe51b2800 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- 172.21.15.191:0/2329686925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 msgr2=0x55cbe549a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/2329686925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55cbe5f83770 tx=0x55cbe52a3b60 comp rx=0 tx=0).stop 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- 172.21.15.191:0/2329686925 shutdown_connections 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/2329686925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- 172.21.15.191:0/2329686925 >> 172.21.15.191:0/2329686925 conn(0x55cbe51b2400 msgr2=0x55cbe529f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- 172.21.15.191:0/2329686925 shutdown_connections 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- 172.21.15.191:0/2329686925 wait complete. 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc225ac3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe52543c0 con 0x55cbe51b2800 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39768/0 (socket says 172.21.15.191:39768) 2024-04-27T04:34:46.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.973+0000 7fc223a6d700 1 -- 172.21.15.191:0/668651768 learned_addr learned my addr 172.21.15.191:0/668651768 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc223a6d700 1 -- 172.21.15.191:0/668651768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51dba00 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc223a6d700 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55cbe5f83e30 tx=0x55cbe52a3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc220a67700 1 -- 172.21.15.191:0/668651768 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b600 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc220a67700 1 -- 172.21.15.191:0/668651768 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe5253c80 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cbe51db860 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc220a67700 1 -- 172.21.15.191:0/668651768 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b1e0 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbe51db6c0 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.974+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc220a67700 1 -- 172.21.15.191:0/668651768 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cbe5f54b00 con 0x55cbe51b2800 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc220a67700 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe51b3400 0x55cbe549b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc225ac3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc225ac3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe5254000 con 0x55cbe51b3800 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc220a67700 1 -- 172.21.15.191:0/668651768 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cbe522d180 con 0x55cbe51b2800 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc22426e700 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe51b3400 0x55cbe549b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39778/0 (socket says 172.21.15.191:39778) 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc223a6d700 1 -- 172.21.15.191:0/3640748122 learned_addr learned my addr 172.21.15.191:0/3640748122 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc223a6d700 1 -- 172.21.15.191:0/3640748122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51da000 con 0x55cbe51b3800 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc22426e700 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe51b3400 0x55cbe549b080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55cbe5f83f20 tx=0x55cbe54fede0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc223a6d700 1 --2- 172.21.15.191:0/3640748122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55cbe6044330 tx=0x55cbe54fe240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0f9a7de494822d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc21e262700 1 -- 172.21.15.191:0/3640748122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b4a0 con 0x55cbe51b3800 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc21e262700 1 -- 172.21.15.191:0/3640748122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe54dc300 con 0x55cbe51b3800 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.975+0000 7fc21e262700 1 -- 172.21.15.191:0/3640748122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b8c0 con 0x55cbe51b3800 2024-04-27T04:34:46.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3640748122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 msgr2=0x55cbe549ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3640748122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55cbe6044330 tx=0x55cbe54fe240 comp rx=0 tx=0).stop 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3640748122 shutdown_connections 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3640748122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3640748122 >> 172.21.15.191:0/3640748122 conn(0x55cbe51b2c00 msgr2=0x55cbe529f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3640748122 shutdown_connections 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3640748122 wait complete. 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc225ac3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe51ee640 con 0x55cbe51b3800 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc223a6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39788/0 (socket says 172.21.15.191:39788) 2024-04-27T04:34:46.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.976+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 learned_addr learned my addr 172.21.15.191:0/3195361555 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51daea0 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc223a6d700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55cbe555ac30 tx=0x55cbe54fefc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc21ca5f700 1 -- 172.21.15.191:0/3195361555 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b4a0 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc21ca5f700 1 -- 172.21.15.191:0/3195361555 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe5253e00 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cbe51da820 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc21ca5f700 1 -- 172.21.15.191:0/3195361555 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b8c0 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.977+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbe51da9c0 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc21ca5f700 1 -- 172.21.15.191:0/3195361555 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cbe5f54dc0 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc21ca5f700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe555ec00 0x55cbe549b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc21ca5f700 1 -- 172.21.15.191:0/3195361555 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cbe522d340 con 0x55cbe51b3800 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc22326c700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe555ec00 0x55cbe549b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cbe51b3c00 0x55cbe549bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555f000 con 0x55cbe51b3c00 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc22426e700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cbe51b3c00 0x55cbe549bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc22326c700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe555ec00 0x55cbe549b600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55cbe5495890 tx=0x55cbe52a3800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.978+0000 7fc22426e700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cbe51b3c00 0x55cbe549bb80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 realm 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555f400 con 0x55cbe51b3c00 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555f800 con 0x55cbe51b3c00 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cbe555f000 0x55cbe549c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555f400 con 0x55cbe555f000 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.979+0000 7fc223a6d700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cbe555f000 0x55cbe549c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc223a6d700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cbe555f000 0x55cbe549c100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc225ac3a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc225ac3a80 20 searching for the correct realm 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6000 con 0x55cbe555f000 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc223a6d700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cbe55e6000 0x55cbe549cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555f400 con 0x55cbe55e6000 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.980+0000 7fc22326c700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cbe55e6000 0x55cbe549cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.981+0000 7fc22326c700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cbe55e6000 0x55cbe549cc00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.981+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.981+0000 7fc22326c700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cbe555f400 0x55cbe549d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.981+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.981+0000 7fc22426e700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cbe555f400 0x55cbe549d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.981+0000 7fc22426e700 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cbe555f400 0x55cbe549d180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.982+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe55e6000 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.983+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe55e6000 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.984+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.985+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe55e6000 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.986+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe55e6000 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.987+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f400 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe51b3c00 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe55e6000 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6400 con 0x55cbe51b3c00 2024-04-27T04:34:46.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.988+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6800 con 0x55cbe51b3c00 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6c00 con 0x55cbe555f000 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 zone default found 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 4 Realm: () 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:46.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 zonegroup default 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7000 con 0x55cbe555f400 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7400 con 0x55cbe555f000 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.989+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6400 con 0x55cbe555f000 2024-04-27T04:34:46.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 20 started sync module instance, tier type = 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.990+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6800 con 0x55cbe55e6000 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.991+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.991+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94334213495552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7000 con 0x55cbe55e6000 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.991+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7c00 con 0x55cbe51b3c00 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.992+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94334213495552] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.992+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.992+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94334213496704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.992+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555f800 con 0x55cbe555f000 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.993+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94334213496704] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.993+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.993+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94334213497856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7800 con 0x55cbe555f000 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.993+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6800 con 0x55cbe55e6000 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.994+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94334213497856] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.994+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.994+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94334213499008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7000 con 0x55cbe55e6000 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.994+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6c00 con 0x55cbe51b3c00 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.995+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.995+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94334213500160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7400 con 0x55cbe51b3c00 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.995+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6400 con 0x55cbe555f400 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.995+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94334213499008] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.996+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.996+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94334213501312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555e400 con 0x55cbe555f400 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.996+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7c00 con 0x55cbe555f400 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.996+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94334213500160] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.997+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.997+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94334213502464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55f2000 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.997+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55f2400 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.997+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94334213501312] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.998+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94334213502464] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.998+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94334213503616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55f2800 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=0 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=1 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=2 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=3 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=4 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=5 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=6 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94334213503616] v31'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 20 add_watcher() i=7 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:45.999+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55f2c00 con 0x55cbe51b3c00 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.000+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.000+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94334214971392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55f3000 con 0x55cbe51b3c00 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94334214971392] v31'100 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc20aa3b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc225ac3a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc225ac3a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc225ac3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.001+0000 7fc225ac3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe51ee780 con 0x55cbe55f3800 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc22326c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc22326c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39802/0 (socket says 172.21.15.191:39802) 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc22326c700 1 -- 172.21.15.191:0/3783362516 learned_addr learned my addr 172.21.15.191:0/3783362516 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc22326c700 1 -- 172.21.15.191:0/3783362516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51dab60 con 0x55cbe55f3800 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc22326c700 1 --2- 172.21.15.191:0/3783362516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55cbe557d680 tx=0x55cbe52a3980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44754e43d0f676d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc20a23a700 1 -- 172.21.15.191:0/3783362516 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe526b760 con 0x55cbe55f3800 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc20a23a700 1 -- 172.21.15.191:0/3783362516 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe560e000 con 0x55cbe55f3800 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.002+0000 7fc20a23a700 1 -- 172.21.15.191:0/3783362516 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe55f8160 con 0x55cbe55f3800 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3783362516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 msgr2=0x55cbe549c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3783362516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55cbe557d680 tx=0x55cbe52a3980 comp rx=0 tx=0).stop 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3783362516 shutdown_connections 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3783362516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3783362516 >> 172.21.15.191:0/3783362516 conn(0x55cbe55f3400 msgr2=0x55cbe52a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3783362516 shutdown_connections 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3783362516 wait complete. 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 Processor -- start 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- start start 2024-04-27T04:34:46.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc225ac3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe5254000 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc22326c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc22326c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:39806/0 (socket says 172.21.15.191:39806) 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.003+0000 7fc22326c700 1 -- 172.21.15.191:0/1703183027 learned_addr learned my addr 172.21.15.191:0/1703183027 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc22326c700 1 -- 172.21.15.191:0/1703183027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe51dad00 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc22326c700 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55cbe55ff260 tx=0x55cbe52a38c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc208a37700 1 -- 172.21.15.191:0/1703183027 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe55f82c0 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc208a37700 1 -- 172.21.15.191:0/1703183027 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe560e180 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc208a37700 1 -- 172.21.15.191:0/1703183027 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cbe55f8420 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cbe51da340 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.004+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbe51dbd40 con 0x55cbe55f3800 2024-04-27T04:34:46.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.005+0000 7fc208a37700 1 -- 172.21.15.191:0/1703183027 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cbe5f55b80 con 0x55cbe55f3800 2024-04-27T04:34:46.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.005+0000 7fc208a37700 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe55e6c00 0x55cbe549d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.005+0000 7fc208a37700 1 -- 172.21.15.191:0/1703183027 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cbe522d6c0 con 0x55cbe55f3800 2024-04-27T04:34:46.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.005+0000 7fc225ac3a80 5 note: GC not initialized 2024-04-27T04:34:46.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.005+0000 7fc22426e700 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe55e6c00 0x55cbe549d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.005+0000 7fc22426e700 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe55e6c00 0x55cbe549d700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55cbe557dd40 tx=0x55cbe560d1a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.006+0000 7fc206232700 20 reqs_thread_entry: start 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.006+0000 7fc225ac3a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.006+0000 7fc225ac3a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.006+0000 7fc20422e700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:16 2024 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.006+0000 7fc203a0c700 20 reqs_thread_entry: start 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.006+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(62 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(63 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 [pgnls start_epoch 31 out=49b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 [pgnls start_epoch 31 out=49b] v31'11 uv11 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe55e6000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(66 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.007+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(69 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe55e6000 2024-04-27T04:34:46.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(70 [pgnls start_epoch 31 out=49b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.008+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(71 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(72 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(73 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.009+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(76 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(77 [pgnls start_epoch 31 out=127b] v31'1 uv1 ondisk = 1) v8 ==== 144+0+127 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(78 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(79 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.010+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe55e6000 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(80 [pgnls start_epoch 31 out=70b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+70 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(81 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(82 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(83 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe55e6000 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.011+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(84 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(85 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(86 [pgnls start_epoch 31 out=49b] v31'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(87 [pgnls start_epoch 31 out=49b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(88 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.012+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe55e6000 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(89 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f400 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(90 [pgnls start_epoch 31 out=49b] v31'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe51b3c00 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(91 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe555f000 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(92 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.013+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555fc00 con 0x55cbe55e6000 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.014+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(93 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.014+0000 7fc225ac3a80 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.014+0000 7fc225ac3a80 20 RGWRados::pool_iterate: got myfoo 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.014+0000 7fc20aa3b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.014+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94334214971392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe555e400 con 0x55cbe51b3c00 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.015+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94334214971392] v31'101 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.015+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94334213495552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e7400 con 0x55cbe55e6000 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.016+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94334213495552] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.016+0000 7fc225ac3a80 20 remove_watcher() i=0 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.016+0000 7fc225ac3a80 2 removed watcher, disabling cache 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.016+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94334213496704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe55e6400 con 0x55cbe51b3c00 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.017+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94334213496704] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.017+0000 7fc225ac3a80 20 remove_watcher() i=1 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.017+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94334213497856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe559e000 con 0x55cbe555f000 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.018+0000 7fc223a6d700 1 -- 172.21.15.191:0/3195361555 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94334213497856] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb200 con 0x55cbe555f000 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.018+0000 7fc225ac3a80 20 remove_watcher() i=2 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.018+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94334213499008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe559e400 con 0x55cbe55e6000 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.019+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94334213499008] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cb440 con 0x55cbe55e6000 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.019+0000 7fc225ac3a80 20 remove_watcher() i=3 2024-04-27T04:34:46.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.019+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94334213500160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe559e800 con 0x55cbe51b3c00 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.020+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94334213500160] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe51b3c00 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.020+0000 7fc225ac3a80 20 remove_watcher() i=4 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.020+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94334213501312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe559ec00 con 0x55cbe555f400 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.021+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94334213501312] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.021+0000 7fc225ac3a80 20 remove_watcher() i=5 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.021+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94334213502464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe559f000 con 0x55cbe555f400 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.022+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94334213502464] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.022+0000 7fc225ac3a80 20 remove_watcher() i=6 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.022+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94334213503616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cbe559f400 con 0x55cbe555f400 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc22426e700 1 -- 172.21.15.191:0/3195361555 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94334213503616] v31'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cbe55cab40 con 0x55cbe555f400 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 20 remove_watcher() i=7 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cbe555f400 msgr2=0x55cbe549d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cbe555f400 0x55cbe549d180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cbe51b3c00 msgr2=0x55cbe549bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cbe51b3c00 0x55cbe549bb80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cbe55e6000 msgr2=0x55cbe549cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cbe55e6000 0x55cbe549cc00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cbe555f000 msgr2=0x55cbe549c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cbe555f000 0x55cbe549c100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe555ec00 msgr2=0x55cbe549b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe555ec00 0x55cbe549b600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55cbe5495890 tx=0x55cbe52a3800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 msgr2=0x55cbe549ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/3195361555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b3800 0x55cbe549ab00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55cbe555ac30 tx=0x55cbe54fefc0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 reap_dead start 2024-04-27T04:34:46.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc22326c700 1 -- 172.21.15.191:0/3195361555 reap_dead start 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 shutdown_connections 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 >> 172.21.15.191:0/3195361555 conn(0x55cbe51b2c00 msgr2=0x55cbe529f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 shutdown_connections 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.023+0000 7fc225ac3a80 1 -- 172.21.15.191:0/3195361555 wait complete. 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe55e6c00 msgr2=0x55cbe549d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe55e6c00 0x55cbe549d700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55cbe557dd40 tx=0x55cbe560d1a0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 msgr2=0x55cbe549c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55cbe55ff260 tx=0x55cbe52a38c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 shutdown_connections 2024-04-27T04:34:46.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe55e6c00 0x55cbe549d700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/1703183027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe55f3800 0x55cbe549c680 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 >> 172.21.15.191:0/1703183027 conn(0x55cbe55f3400 msgr2=0x55cbe52a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 shutdown_connections 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/1703183027 wait complete. 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe51b3400 msgr2=0x55cbe549b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe51b3400 0x55cbe549b080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55cbe5f83f20 tx=0x55cbe54fede0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 msgr2=0x55cbe549a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55cbe5f83e30 tx=0x55cbe52a3a40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 shutdown_connections 2024-04-27T04:34:46.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cbe51b3400 0x55cbe549b080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 --2- 172.21.15.191:0/668651768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbe51b2800 0x55cbe549a580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 >> 172.21.15.191:0/668651768 conn(0x55cbe51b2400 msgr2=0x55cbe529f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 shutdown_connections 2024-04-27T04:34:46.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.024+0000 7fc225ac3a80 1 -- 172.21.15.191:0/668651768 wait complete. 2024-04-27T04:34:46.071 INFO:teuthology.orchestra.run.smithi191.stdout:[ 2024-04-27T04:34:46.072 INFO:teuthology.orchestra.run.smithi191.stdout: "myfoo" 2024-04-27T04:34:46.072 INFO:teuthology.orchestra.run.smithi191.stdout:] 2024-04-27T04:34:46.072 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-27T04:34:46.082 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.097 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.106 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.151 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-04-27T04:34:46.185 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.216 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.249 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.250 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T04:34:46.250 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T04:34:46.250 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T04:34:46.295 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:46.296 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f6076a04a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f6076a04a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d8cfc9f500 con 0x55d8cf974c00 2024-04-27T04:34:46.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f6076a04a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d8cfa16a00 con 0x55d8cf974800 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f60749ae700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f60741ad700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55d8cf974c00 0x55d8cfa60800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60336/0 (socket says 172.21.15.191:60336) 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f60741ad700 1 -- 172.21.15.191:0/1606609237 learned_addr learned my addr 172.21.15.191:0/1606609237 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f60749ae700 1 -- 172.21.15.191:0/1606609237 >> v1:172.21.15.138:6789/0 conn(0x55d8cf974c00 legacy=0x55d8cfa60800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.294+0000 7f60749ae700 1 -- 172.21.15.191:0/1606609237 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99dd40 con 0x55d8cf974800 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f60749ae700 1 --2- 172.21.15.191:0/1606609237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55d8cfc579e0 tx=0x55d8cfa65920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78682ca41d2ec03 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f60739ac700 1 -- 172.21.15.191:0/1606609237 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d4a0 con 0x55d8cf974800 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f60739ac700 1 -- 172.21.15.191:0/1606609237 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfc9f800 con 0x55d8cf974800 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f60739ac700 1 -- 172.21.15.191:0/1606609237 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d600 con 0x55d8cf974800 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 -- 172.21.15.191:0/1606609237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 msgr2=0x55d8cfc5c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1606609237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55d8cfc579e0 tx=0x55d8cfa65920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 -- 172.21.15.191:0/1606609237 shutdown_connections 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1606609237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 -- 172.21.15.191:0/1606609237 >> 172.21.15.191:0/1606609237 conn(0x55d8cf974400 msgr2=0x55d8cfa60000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 -- 172.21.15.191:0/1606609237 shutdown_connections 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.295+0000 7f6076a04a80 1 -- 172.21.15.191:0/1606609237 wait complete. 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f6076a04a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f6076a04a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cfa16500 con 0x55d8cf974800 2024-04-27T04:34:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f60749ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f60749ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46838/0 (socket says 172.21.15.191:46838) 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.296+0000 7f60749ae700 1 -- 172.21.15.191:0/2930141805 learned_addr learned my addr 172.21.15.191:0/2930141805 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f60749ae700 1 -- 172.21.15.191:0/2930141805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99dba0 con 0x55d8cf974800 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f60749ae700 1 --2- 172.21.15.191:0/2930141805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55d8cfae40c0 tx=0x55d8cfa65800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a209b7bdcfd7c8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f60731ab700 1 -- 172.21.15.191:0/2930141805 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d760 con 0x55d8cf974800 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f60731ab700 1 -- 172.21.15.191:0/2930141805 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfc9fb00 con 0x55d8cf974800 2024-04-27T04:34:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f60731ab700 1 -- 172.21.15.191:0/2930141805 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d4a0 con 0x55d8cf974800 2024-04-27T04:34:46.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 -- 172.21.15.191:0/2930141805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 msgr2=0x55d8cfc5c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2930141805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55d8cfae40c0 tx=0x55d8cfa65800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 -- 172.21.15.191:0/2930141805 shutdown_connections 2024-04-27T04:34:46.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2930141805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 -- 172.21.15.191:0/2930141805 >> 172.21.15.191:0/2930141805 conn(0x55d8cf974400 msgr2=0x55d8cfa61000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 -- 172.21.15.191:0/2930141805 shutdown_connections 2024-04-27T04:34:46.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 -- 172.21.15.191:0/2930141805 wait complete. 2024-04-27T04:34:46.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.297+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f6076a04a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f6076a04a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cfa163c0 con 0x55d8cf974800 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60749ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60749ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46840/0 (socket says 172.21.15.191:46840) 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60749ae700 1 -- 172.21.15.191:0/2813378372 learned_addr learned my addr 172.21.15.191:0/2813378372 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60749ae700 1 -- 172.21.15.191:0/2813378372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99da00 con 0x55d8cf974800 2024-04-27T04:34:46.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60749ae700 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55d8cfae5980 tx=0x55d8cfa65920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60719a8700 1 -- 172.21.15.191:0/2813378372 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d600 con 0x55d8cf974800 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60719a8700 1 -- 172.21.15.191:0/2813378372 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfc9fc80 con 0x55d8cf974800 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d8cf99d860 con 0x55d8cf974800 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f60719a8700 1 -- 172.21.15.191:0/2813378372 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d8c0 con 0x55d8cf974800 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.298+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8cf99d6c0 con 0x55d8cf974800 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60719a8700 1 -- 172.21.15.191:0/2813378372 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d8d0716b00 con 0x55d8cf974800 2024-04-27T04:34:46.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f6076a04a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f6076a04a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cfa16000 con 0x55d8cf975400 2024-04-27T04:34:46.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60719a8700 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfe000 0x55d8cfc5d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60719a8700 1 -- 172.21.15.191:0/2813378372 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d8cf9ef180 con 0x55d8cf974800 2024-04-27T04:34:46.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60749ae700 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfe000 0x55d8cfc5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60751af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60751af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46842/0 (socket says 172.21.15.191:46842) 2024-04-27T04:34:46.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.299+0000 7f60751af700 1 -- 172.21.15.191:0/2500388617 learned_addr learned my addr 172.21.15.191:0/2500388617 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f60751af700 1 -- 172.21.15.191:0/2500388617 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99c000 con 0x55d8cf975400 2024-04-27T04:34:46.330 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f60749ae700 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfe000 0x55d8cfc5d080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55d8d0804ed0 tx=0x55d8cfcc2b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f60751af700 1 --2- 172.21.15.191:0/2500388617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55d8d0804210 tx=0x55d8cfcc2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7166ce5463c412 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f606f1a3700 1 -- 172.21.15.191:0/2500388617 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d600 con 0x55d8cf975400 2024-04-27T04:34:46.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f606f1a3700 1 -- 172.21.15.191:0/2500388617 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfa15c80 con 0x55d8cf975400 2024-04-27T04:34:46.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f606f1a3700 1 -- 172.21.15.191:0/2500388617 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d8c0 con 0x55d8cf975400 2024-04-27T04:34:46.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 -- 172.21.15.191:0/2500388617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 msgr2=0x55d8cfc5cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2500388617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55d8d0804210 tx=0x55d8cfcc2e40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 -- 172.21.15.191:0/2500388617 shutdown_connections 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2500388617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 -- 172.21.15.191:0/2500388617 >> 172.21.15.191:0/2500388617 conn(0x55d8cf974c00 msgr2=0x55d8cfa61800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 -- 172.21.15.191:0/2500388617 shutdown_connections 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.300+0000 7f6076a04a80 1 -- 172.21.15.191:0/2500388617 wait complete. 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f6076a04a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f6076a04a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cf9b0640 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f60751af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f60751af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46848/0 (socket says 172.21.15.191:46848) 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 learned_addr learned my addr 172.21.15.191:0/1156757630 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99cea0 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f60751af700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55d8d0804810 tx=0x55d8cfcc2a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f606d9a0700 1 -- 172.21.15.191:0/1156757630 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d760 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f606d9a0700 1 -- 172.21.15.191:0/1156757630 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfa15b00 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d8cf99c820 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f606d9a0700 1 -- 172.21.15.191:0/1156757630 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfa2d600 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.301+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8cf99c9c0 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.302+0000 7f606d9a0700 1 -- 172.21.15.191:0/1156757630 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d8d0716dc0 con 0x55d8cf975400 2024-04-27T04:34:46.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.302+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.302+0000 7f606d9a0700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfec00 0x55d8cfc5d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.302+0000 7f606d9a0700 1 -- 172.21.15.191:0/1156757630 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d8cf9ef340 con 0x55d8cf975400 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f60741ad700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfec00 0x55d8cfc5d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d8cf975800 0x55d8cfc5db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cf975c00 con 0x55d8cf975800 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f60751af700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d8cf975800 0x55d8cfc5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f60741ad700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfec00 0x55d8cfc5d600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55d8cfcfdcb0 tx=0x55d8cfdb40c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f60751af700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d8cf975800 0x55d8cfc5db80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 20 realm 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff000 con 0x55d8cf975800 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.303+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff400 con 0x55d8cf975800 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff800 con 0x55d8cf975800 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d8cfcffc00 0x55d8cfc5e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff000 con 0x55d8cfcffc00 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f60749ae700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d8cfcffc00 0x55d8cfc5e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.304+0000 7f60749ae700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d8cfcffc00 0x55d8cfc5e100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f6076a04a80 20 searching for the correct realm 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff400 con 0x55d8cfcffc00 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f60749ae700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d8cfcff400 0x55d8cfc5ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff000 con 0x55d8cfcff400 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f60741ad700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d8cfcff400 0x55d8cfc5ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.305+0000 7f60741ad700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d8cfcff400 0x55d8cfc5ec00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60741ad700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d8cfcff000 0x55d8cfc5f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60751af700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d8cfcff000 0x55d8cfc5f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60751af700 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d8cfcff000 0x55d8cfc5f180 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.306+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.307+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff400 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.308+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff400 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.309+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.310+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.311+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.311+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.311+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.311+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff400 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.312+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff400 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcffc00 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.313+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff400 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cf975c00 con 0x55d8cf975800 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca400 con 0x55d8cf975800 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca800 con 0x55d8cfcffc00 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 zone default found 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 4 Realm: () 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:46.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 zonegroup default 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.314+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcac00 con 0x55d8cfcff000 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb000 con 0x55d8cfcffc00 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb400 con 0x55d8cfcffc00 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.344 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 started sync module instance, tier type = 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.315+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb800 con 0x55d8cfcff400 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.316+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.316+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94389687106304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca800 con 0x55d8cfcff400 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.316+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb000 con 0x55d8cf975800 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.317+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94389687106304] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.317+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.317+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94389687107456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb400 con 0x55d8cf975800 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.317+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca400 con 0x55d8cfcffc00 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.318+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94389687107456] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.319+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.319+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94389687108608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcac00 con 0x55d8cfcffc00 2024-04-27T04:34:46.345 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.319+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cf975c00 con 0x55d8cfcff400 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.320+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94389687108608] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.320+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.320+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94389687109760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfcff800 con 0x55d8cfcff400 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.320+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cf975800 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.321+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.321+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94389687110912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb800 con 0x55d8cf975800 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.321+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca800 con 0x55d8cfcff000 2024-04-27T04:34:46.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.321+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94389687109760] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.323+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94389687110912] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.324+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.324+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94389687112064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcbc00 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.324+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb000 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.324+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94389687112064] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94389687113216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb400 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca000 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94389687113216] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94389687114368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb800 con 0x55d8cfcff000 2024-04-27T04:34:46.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=0 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=1 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=2 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=3 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=4 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=5 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.325+0000 7f6076a04a80 20 add_watcher() i=6 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.326+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94389687114368] v31'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.326+0000 7f6076a04a80 20 add_watcher() i=7 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.326+0000 7f6076a04a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.326+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca800 con 0x55d8cf975800 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.327+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.327+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94389688672256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcbc00 con 0x55d8cf975800 2024-04-27T04:34:46.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94389688672256] v31'102 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f605b97c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f6076a04a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f6076a04a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f6076a04a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f6076a04a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cf9b0780 con 0x55d8cfdcb400 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f60741ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f60741ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46856/0 (socket says 172.21.15.191:46856) 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.328+0000 7f60741ad700 1 -- 172.21.15.191:0/1771703496 learned_addr learned my addr 172.21.15.191:0/1771703496 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f60741ad700 1 -- 172.21.15.191:0/1771703496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99cb60 con 0x55d8cfdcb400 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f60741ad700 1 --2- 172.21.15.191:0/1771703496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55d8cfd655f0 tx=0x55d8cfcc3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd79ba3aaf7f27aa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f605b17b700 1 -- 172.21.15.191:0/1771703496 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfde8000 con 0x55d8cfdcb400 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f605b17b700 1 -- 172.21.15.191:0/1771703496 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfdec000 con 0x55d8cfdcb400 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f605b17b700 1 -- 172.21.15.191:0/1771703496 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfde8160 con 0x55d8cfdcb400 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f6076a04a80 1 -- 172.21.15.191:0/1771703496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 msgr2=0x55d8cfc5e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.329+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1771703496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55d8cfd655f0 tx=0x55d8cfcc3f80 comp rx=0 tx=0).stop 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 -- 172.21.15.191:0/1771703496 shutdown_connections 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1771703496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 -- 172.21.15.191:0/1771703496 >> 172.21.15.191:0/1771703496 conn(0x55d8cfdcb000 msgr2=0x55d8cfa62000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 -- 172.21.15.191:0/1771703496 shutdown_connections 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 -- 172.21.15.191:0/1771703496 wait complete. 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 Processor -- start 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 -- start start 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f6076a04a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cfa16000 con 0x55d8cfdcb400 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f60741ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f60741ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46872/0 (socket says 172.21.15.191:46872) 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f60741ad700 1 -- 172.21.15.191:0/1326045856 learned_addr learned my addr 172.21.15.191:0/1326045856 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.330+0000 7f60741ad700 1 -- 172.21.15.191:0/1326045856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cf99cd00 con 0x55d8cfdcb400 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f60741ad700 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55d8cfdea210 tx=0x55d8cfdb44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6059978700 1 -- 172.21.15.191:0/1326045856 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfde82c0 con 0x55d8cfdcb400 2024-04-27T04:34:46.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6059978700 1 -- 172.21.15.191:0/1326045856 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cfdec180 con 0x55d8cfdcb400 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d8cf99c340 con 0x55d8cfdcb400 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6059978700 1 -- 172.21.15.191:0/1326045856 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d8cfde8000 con 0x55d8cfdcb400 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8cf99cb60 con 0x55d8cfdcb400 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6059978700 1 -- 172.21.15.191:0/1326045856 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d8d0717b80 con 0x55d8cfdcb400 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6059978700 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfe6c000 0x55d8cfc5f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.331+0000 7f6059978700 1 -- 172.21.15.191:0/1326045856 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d8cf9ef500 con 0x55d8cfdcb400 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.332+0000 7f6076a04a80 5 note: GC not initialized 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.332+0000 7f60751af700 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfe6c000 0x55d8cfc5f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.332+0000 7f60751af700 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfe6c000 0x55d8cfc5f700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55d8cfd65ec0 tx=0x55d8cfdb4360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.332+0000 7f6057173700 20 reqs_thread_entry: start 2024-04-27T04:34:46.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f605516f700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:16 2024 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f605494d700 20 reqs_thread_entry: start 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcb800 con 0x55d8cfcff400 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.333+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca800 con 0x55d8cfcff000 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6c400 con 0x55d8cfcff400 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.334+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6c800 con 0x55d8cfcff000 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(65 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:46.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 4.25 4:a741c040:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6cc00 con 0x55d8cfcffc00 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 4.39 4:9e5e97cb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d000 con 0x55d8cfcffc00 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 4.1c 4:381df985:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d400 con 0x55d8cfcff000 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 4.1c 4:3a1291c0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d800 con 0x55d8cfcff000 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 4.5 4:a135d554:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6dc00 con 0x55d8cfcffc00 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 4.2a 4:57f23c0a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6cc00 con 0x55d8cf975800 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d000 con 0x55d8cf975800 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 4.11 4:8b5d38c9:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca400 con 0x55d8cfcff000 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:74 4.29 4:95afba01:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcac00 con 0x55d8cfcffc00 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.2e 4:74385e64:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6c800 con 0x55d8cfcffc00 2024-04-27T04:34:46.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 4.1f 4:fa83b5f2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d400 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(76 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.335+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.336+0000 7f605b97c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.336+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94389688672256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d800 con 0x55d8cf975800 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.337+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94389688672256] v31'103 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.337+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94389687106304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6cc00 con 0x55d8cfcff400 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.338+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94389687106304] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.338+0000 7f6076a04a80 20 remove_watcher() i=0 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.338+0000 7f6076a04a80 2 removed watcher, disabling cache 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.338+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94389687107456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d000 con 0x55d8cf975800 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.339+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94389687107456] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.339+0000 7f6076a04a80 20 remove_watcher() i=1 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.339+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94389687108608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdcbc00 con 0x55d8cfcffc00 2024-04-27T04:34:46.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.340+0000 7f60749ae700 1 -- 172.21.15.191:0/1156757630 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94389687108608] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5200 con 0x55d8cfcffc00 2024-04-27T04:34:46.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.340+0000 7f6076a04a80 20 remove_watcher() i=2 2024-04-27T04:34:46.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.340+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94389687109760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfdca800 con 0x55d8cfcff400 2024-04-27T04:34:46.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.341+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94389687109760] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda5440 con 0x55d8cfcff400 2024-04-27T04:34:46.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.341+0000 7f6076a04a80 20 remove_watcher() i=3 2024-04-27T04:34:46.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.341+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94389687110912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6dc00 con 0x55d8cf975800 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.342+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94389687110912] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cf975800 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.342+0000 7f6076a04a80 20 remove_watcher() i=4 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.342+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94389687112064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6c800 con 0x55d8cfcff000 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.343+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94389687112064] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.343+0000 7f6076a04a80 20 remove_watcher() i=5 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.343+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94389687113216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6d400 con 0x55d8cfcff000 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.344+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94389687113216] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.344+0000 7f6076a04a80 20 remove_watcher() i=6 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.344+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94389687114368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d8cfe6c400 con 0x55d8cfcff000 2024-04-27T04:34:46.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f60751af700 1 -- 172.21.15.191:0/1156757630 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94389687114368] v31'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d8cfda4b40 con 0x55d8cfcff000 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 20 remove_watcher() i=7 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d8cfcff000 msgr2=0x55d8cfc5f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d8cfcff000 0x55d8cfc5f180 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d8cf975800 msgr2=0x55d8cfc5db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d8cf975800 0x55d8cfc5db80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d8cfcff400 msgr2=0x55d8cfc5ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d8cfcff400 0x55d8cfc5ec00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d8cfcffc00 msgr2=0x55d8cfc5e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d8cfcffc00 0x55d8cfc5e100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfec00 msgr2=0x55d8cfc5d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfec00 0x55d8cfc5d600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55d8cfcfdcb0 tx=0x55d8cfdb40c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 msgr2=0x55d8cfc5cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1156757630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf975400 0x55d8cfc5cb00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55d8d0804810 tx=0x55d8cfcc2a20 comp rx=0 tx=0).stop 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.345+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 reap_dead start 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f60741ad700 1 -- 172.21.15.191:0/1156757630 reap_dead start 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 shutdown_connections 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 >> 172.21.15.191:0/1156757630 conn(0x55d8cf974c00 msgr2=0x55d8cfa61800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 shutdown_connections 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1156757630 wait complete. 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfe6c000 msgr2=0x55d8cfc5f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfe6c000 0x55d8cfc5f700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55d8cfd65ec0 tx=0x55d8cfdb4360 comp rx=0 tx=0).stop 2024-04-27T04:34:46.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 msgr2=0x55d8cfc5e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55d8cfdea210 tx=0x55d8cfdb44e0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 shutdown_connections 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfe6c000 0x55d8cfc5f700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/1326045856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cfdcb400 0x55d8cfc5e680 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 >> 172.21.15.191:0/1326045856 conn(0x55d8cfdcb000 msgr2=0x55d8cfa62000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 shutdown_connections 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/1326045856 wait complete. 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfe000 msgr2=0x55d8cfc5d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfe000 0x55d8cfc5d080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55d8d0804ed0 tx=0x55d8cfcc2b40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 msgr2=0x55d8cfc5c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55d8cfae5980 tx=0x55d8cfa65920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 shutdown_connections 2024-04-27T04:34:46.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d8cfcfe000 0x55d8cfc5d080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 --2- 172.21.15.191:0/2813378372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8cf974800 0x55d8cfc5c580 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.346+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 >> 172.21.15.191:0/2813378372 conn(0x55d8cf974400 msgr2=0x55d8cfa61000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.347+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 shutdown_connections 2024-04-27T04:34:46.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.347+0000 7f6076a04a80 1 -- 172.21.15.191:0/2813378372 wait complete. 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "num_shards": 11, 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "tenant": "", 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "zonegroup": "29d2c105-c074-4639-b257-47b1060771f9", 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_rule": "default-placement", 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "explicit_placement": { 2024-04-27T04:34:46.380 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "", 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "", 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "" 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "marker": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": "Normal", 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "versioned": false, 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "versioning_enabled": false, 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "object_lock_enabled": false, 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_enabled": false, 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:34:46.381 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "mtime": "2024-04-27T04:34:45.736480Z", 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "creation_time": "2024-04-27T04:34:45.731629Z", 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "usage": {}, 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:46.382 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:46.383 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:46.383 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:46.383 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:46.383 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:46.383 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:34:45.736480Z', 'creation_time': '2024-04-27T04:34:45.731629Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T04:34:46.383 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-27T04:34:46.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-27T04:34:46.384 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-27T04:34:46.493 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:46.493 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed9908a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5605e3fc8c00 0x5605e42b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed9908a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5605e42f3500 con 0x5605e3fc8800 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed9908a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5605e406aa00 con 0x5605e3fc8c00 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed70b1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5605e3fc8c00 0x5605e42b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed70b1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5605e3fc8c00 0x5605e42b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46880/0 (socket says 172.21.15.191:46880) 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed70b1700 1 -- 172.21.15.191:0/2654143423 learned_addr learned my addr 172.21.15.191:0/2654143423 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.427+0000 7f5ed78b2700 1 --1- 172.21.15.191:0/2654143423 >> v1:172.21.15.138:6789/0 conn(0x5605e3fc8800 0x5605e40b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60348/0 (socket says 172.21.15.191:60348) 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed70b1700 1 -- 172.21.15.191:0/2654143423 >> v1:172.21.15.138:6789/0 conn(0x5605e3fc8800 legacy=0x5605e40b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed70b1700 1 -- 172.21.15.191:0/2654143423 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff1d40 con 0x5605e3fc8c00 2024-04-27T04:34:46.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/2654143423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8c00 0x5605e42b0580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5605e42ab890 tx=0x5605e40b9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f2400462cb3634 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed68b0700 1 -- 172.21.15.191:0/2654143423 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40814a0 con 0x5605e3fc8c00 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed68b0700 1 -- 172.21.15.191:0/2654143423 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e42f3800 con 0x5605e3fc8c00 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed68b0700 1 -- 172.21.15.191:0/2654143423 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e4081600 con 0x5605e3fc8c00 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2654143423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8c00 msgr2=0x5605e42b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2654143423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8c00 0x5605e42b0580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5605e42ab890 tx=0x5605e40b9920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2654143423 shutdown_connections 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2654143423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8c00 0x5605e42b0580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.428+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2654143423 >> 172.21.15.191:0/2654143423 conn(0x5605e3fc8400 msgr2=0x5605e40b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2654143423 shutdown_connections 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2654143423 wait complete. 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed9908a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed9908a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e406a500 con 0x5605e3fc8800 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46888/0 (socket says 172.21.15.191:46888) 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.429+0000 7f5ed78b2700 1 -- 172.21.15.191:0/2096126190 learned_addr learned my addr 172.21.15.191:0/2096126190 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed78b2700 1 -- 172.21.15.191:0/2096126190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff1ba0 con 0x5605e3fc8800 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/2096126190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5605e4139890 tx=0x5605e40b9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e03ff995a2ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed60af700 1 -- 172.21.15.191:0/2096126190 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40811e0 con 0x5605e3fc8800 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed60af700 1 -- 172.21.15.191:0/2096126190 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e42f3b00 con 0x5605e3fc8800 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed60af700 1 -- 172.21.15.191:0/2096126190 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40814a0 con 0x5605e3fc8800 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2096126190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 msgr2=0x5605e42b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2096126190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5605e4139890 tx=0x5605e40b9b60 comp rx=0 tx=0).stop 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2096126190 shutdown_connections 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2096126190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2096126190 >> 172.21.15.191:0/2096126190 conn(0x5605e3fc8400 msgr2=0x5605e40b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2096126190 shutdown_connections 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.430+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2096126190 wait complete. 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed9908a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed9908a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e406a3c0 con 0x5605e3fc8800 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46904/0 (socket says 172.21.15.191:46904) 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed78b2700 1 -- 172.21.15.191:0/1910361667 learned_addr learned my addr 172.21.15.191:0/1910361667 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed78b2700 1 -- 172.21.15.191:0/1910361667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff1a00 con 0x5605e3fc8800 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5605e41396b0 tx=0x5605e40b9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.431+0000 7f5ed48ac700 1 -- 172.21.15.191:0/1910361667 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e4081600 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed48ac700 1 -- 172.21.15.191:0/1910361667 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e4069c80 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605e3ff1860 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed48ac700 1 -- 172.21.15.191:0/1910361667 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40811e0 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e3ff16c0 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed48ac700 1 -- 172.21.15.191:0/1910361667 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5605e4d6ab00 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed48ac700 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e3fc9400 0x5605e42b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed9908a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed9908a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e406a000 con 0x5605e3fc9800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed48ac700 1 -- 172.21.15.191:0/1910361667 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5605e4043180 con 0x5605e3fc8800 2024-04-27T04:34:46.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e3fc9400 0x5605e42b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.432+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46906/0 (socket says 172.21.15.191:46906) 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed78b2700 1 -- 172.21.15.191:0/2891635165 learned_addr learned my addr 172.21.15.191:0/2891635165 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed78b2700 1 -- 172.21.15.191:0/2891635165 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff0000 con 0x5605e3fc9800 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e3fc9400 0x5605e42b1080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5605e4e3aae0 tx=0x5605e4312de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/2891635165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5605e4e3b0e0 tx=0x5605e4312240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ccaca02781cc0d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed20a7700 1 -- 172.21.15.191:0/2891635165 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40814a0 con 0x5605e3fc9800 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed20a7700 1 -- 172.21.15.191:0/2891635165 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e42f2300 con 0x5605e3fc9800 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed20a7700 1 -- 172.21.15.191:0/2891635165 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40818c0 con 0x5605e3fc9800 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2891635165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 msgr2=0x5605e42b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2891635165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5605e4e3b0e0 tx=0x5605e4312240 comp rx=0 tx=0).stop 2024-04-27T04:34:46.499 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2891635165 shutdown_connections 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2891635165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2891635165 >> 172.21.15.191:0/2891635165 conn(0x5605e3fc8c00 msgr2=0x5605e40b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2891635165 shutdown_connections 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.433+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2891635165 wait complete. 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed9908a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed9908a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e4004640 con 0x5605e3fc9800 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed78b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46918/0 (socket says 172.21.15.191:46918) 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 learned_addr learned my addr 172.21.15.191:0/19547312 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.434+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff0ea0 con 0x5605e3fc9800 2024-04-27T04:34:46.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5605e4e3a360 tx=0x5605e4312fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed08a4700 1 -- 172.21.15.191:0/19547312 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40814a0 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed08a4700 1 -- 172.21.15.191:0/19547312 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e4069e00 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605e3ff0820 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed08a4700 1 -- 172.21.15.191:0/19547312 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e40818c0 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e3ff09c0 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.435+0000 7f5ed08a4700 1 -- 172.21.15.191:0/19547312 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5605e4d6adc0 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed08a4700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e4372c00 0x5605e42b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed08a4700 1 -- 172.21.15.191:0/19547312 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5605e4043340 con 0x5605e3fc9800 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e4372c00 0x5605e42b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605e3fc9c00 0x5605e42b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373000 con 0x5605e3fc9c00 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605e3fc9c00 0x5605e42b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e4372c00 0x5605e42b1600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5605e42ab890 tx=0x5605e40b9800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.436+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605e3fc9c00 0x5605e42b1b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 realm 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373400 con 0x5605e3fc9c00 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:46.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373800 con 0x5605e3fc9c00 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373c00 con 0x5605e3fc9c00 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605e4373000 0x5605e42b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373400 con 0x5605e4373000 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.437+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605e4373000 0x5605e42b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605e4373000 0x5605e42b2100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed9908a80 20 searching for the correct realm 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6000 con 0x5605e4373000 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed78b2700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605e43f6000 0x5605e42b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373400 con 0x5605e43f6000 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.438+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605e43f6000 0x5605e42b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.439+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605e43f6000 0x5605e42b2c00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.439+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.439+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605e4373400 0x5605e42b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.439+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.439+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605e4373400 0x5605e42b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.439+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605e4373400 0x5605e42b3180 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.440+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e43f6000 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.441+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e43f6000 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.442+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.443+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e43f6000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.444+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e43f6000 2024-04-27T04:34:46.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.445+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373400 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e43f6000 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:46.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6400 con 0x5605e3fc9c00 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.446+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6800 con 0x5605e3fc9c00 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6c00 con 0x5605e4373000 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 zone default found 2024-04-27T04:34:46.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 4 Realm: () 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 zonegroup default 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7000 con 0x5605e4373400 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.447+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7400 con 0x5605e4373000 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6400 con 0x5605e4373000 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 20 started sync module instance, tier type = 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.448+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6800 con 0x5605e43f6000 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.449+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.449+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94583302814464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7000 con 0x5605e43f6000 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.449+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7c00 con 0x5605e3fc9c00 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.450+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94583302814464] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.450+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.450+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94583302815616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373c00 con 0x5605e3fc9c00 2024-04-27T04:34:46.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.450+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373800 con 0x5605e4373000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.451+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.451+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94583302816768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e4373000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.451+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6800 con 0x5605e43f6000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.451+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94583302815616] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.452+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94583302816768] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.452+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.452+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94583302817920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7000 con 0x5605e43f6000 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.452+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6c00 con 0x5605e3fc9c00 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.453+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.453+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94583302819072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7400 con 0x5605e3fc9c00 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.453+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6400 con 0x5605e4373400 2024-04-27T04:34:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.453+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94583302817920] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.454+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94583302819072] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.455+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.455+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94583302820224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4372400 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.455+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe000 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.456+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.456+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94583302821376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe400 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.456+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe800 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.456+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94583302820224] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94583302822528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fec00 con 0x5605e4373400 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=0 2024-04-27T04:34:46.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=1 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=2 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=3 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=4 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=5 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94583302821376] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=6 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94583302822528] v31'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 20 add_watcher() i=7 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.457+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43ff000 con 0x5605e3fc9c00 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.458+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.458+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94583304232960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43ff400 con 0x5605e3fc9c00 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94583304232960] v31'104 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ebe880700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed9908a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed9908a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed9908a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.459+0000 7f5ed9908a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e4004780 con 0x5605e43ffc00 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ed70b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ed70b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46926/0 (socket says 172.21.15.191:46926) 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ed70b1700 1 -- 172.21.15.191:0/3109931945 learned_addr learned my addr 172.21.15.191:0/3109931945 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ed70b1700 1 -- 172.21.15.191:0/3109931945 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff0b60 con 0x5605e43ffc00 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/3109931945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5605e437f6e0 tx=0x5605e40b9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13df7529178f846 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ebe07f700 1 -- 172.21.15.191:0/3109931945 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e4081760 con 0x5605e43ffc00 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ebe07f700 1 -- 172.21.15.191:0/3109931945 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e4418000 con 0x5605e43ffc00 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.460+0000 7f5ebe07f700 1 -- 172.21.15.191:0/3109931945 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e4400160 con 0x5605e43ffc00 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 -- 172.21.15.191:0/3109931945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 msgr2=0x5605e42b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/3109931945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5605e437f6e0 tx=0x5605e40b9980 comp rx=0 tx=0).stop 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 -- 172.21.15.191:0/3109931945 shutdown_connections 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/3109931945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 -- 172.21.15.191:0/3109931945 >> 172.21.15.191:0/3109931945 conn(0x5605e43ff800 msgr2=0x5605e40b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 -- 172.21.15.191:0/3109931945 shutdown_connections 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.461+0000 7f5ed9908a80 1 -- 172.21.15.191:0/3109931945 wait complete. 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed9908a80 1 Processor -- start 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed9908a80 1 -- start start 2024-04-27T04:34:46.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed9908a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed9908a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e406a000 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed70b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed70b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46938/0 (socket says 172.21.15.191:46938) 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed70b1700 1 -- 172.21.15.191:0/2612770767 learned_addr learned my addr 172.21.15.191:0/2612770767 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed70b1700 1 -- 172.21.15.191:0/2612770767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3ff0d00 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ed70b1700 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x5605e44067e0 tx=0x5605e40b98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.462+0000 7f5ebc87c700 1 -- 172.21.15.191:0/2612770767 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e44002c0 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ebc87c700 1 -- 172.21.15.191:0/2612770767 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605e4418180 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605e3ff0340 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ebc87c700 1 -- 172.21.15.191:0/2612770767 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605e4400420 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e3ff1d40 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ebc87c700 1 -- 172.21.15.191:0/2612770767 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5605e4d6bb80 con 0x5605e43ffc00 2024-04-27T04:34:46.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ebc87c700 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e43f7c00 0x5605e42b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.463+0000 7f5ebc87c700 1 -- 172.21.15.191:0/2612770767 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5605e40436c0 con 0x5605e43ffc00 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.464+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e43f7c00 0x5605e42b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.464+0000 7f5ed9908a80 5 note: GC not initialized 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.464+0000 7f5ed80b3700 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e43f7c00 0x5605e42b3700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5605e437ff20 tx=0x5605e44252c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.464+0000 7f5eba077700 20 reqs_thread_entry: start 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.464+0000 7f5ed9908a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5eb8073700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:16 2024 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5eb7851700 20 reqs_thread_entry: start 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4372400 con 0x5605e3fc9c00 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6c00 con 0x5605e43f6000 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.465+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7400 con 0x5605e43f6000 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f6400 con 0x5605e4373400 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(65 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 4.25 4:a741c040:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4372400 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 4.39 4:9e5e97cb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe000 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.466+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 4.1c 4:381df985:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe400 con 0x5605e4373400 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 4.1c 4:3a1291c0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe800 con 0x5605e4373400 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 4.5 4:a135d554:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fec00 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 4.2a 4:57f23c0a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43ff000 con 0x5605e3fc9c00 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43ff400 con 0x5605e3fc9c00 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 4.11 4:8b5d38c9:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fe000 con 0x5605e4373400 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:74 4.29 4:95afba01:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43fec00 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.2e 4:74385e64:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4373800 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 4.1f 4:fa83b5f2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e4372400 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.467+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(76 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.468+0000 7f5ebe880700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.468+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94583304232960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43f7800 con 0x5605e3fc9c00 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.469+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94583304232960] v31'105 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.469+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94583302814464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e0000 con 0x5605e43f6000 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.470+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94583302814464] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.470+0000 7f5ed9908a80 20 remove_watcher() i=0 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.470+0000 7f5ed9908a80 2 removed watcher, disabling cache 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.470+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94583302815616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e0400 con 0x5605e3fc9c00 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.471+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94583302815616] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.471+0000 7f5ed9908a80 20 remove_watcher() i=1 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.471+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94583302816768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e0800 con 0x5605e4373000 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.472+0000 7f5ed78b2700 1 -- 172.21.15.191:0/19547312 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94583302816768] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7200 con 0x5605e4373000 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.472+0000 7f5ed9908a80 20 remove_watcher() i=2 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.472+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94583302817920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e0c00 con 0x5605e43f6000 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.473+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94583302817920] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e7440 con 0x5605e43f6000 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.473+0000 7f5ed9908a80 20 remove_watcher() i=3 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.473+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94583302819072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e1000 con 0x5605e3fc9c00 2024-04-27T04:34:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.474+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94583302819072] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e3fc9c00 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.474+0000 7f5ed9908a80 20 remove_watcher() i=4 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.474+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94583302820224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e1400 con 0x5605e4373400 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.475+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94583302820224] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.475+0000 7f5ed9908a80 20 remove_watcher() i=5 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.475+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94583302821376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e1800 con 0x5605e4373400 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.476+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94583302821376] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.476+0000 7f5ed9908a80 20 remove_watcher() i=6 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.476+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94583302822528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605e43e1c00 con 0x5605e4373400 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed80b3700 1 -- 172.21.15.191:0/19547312 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94583302822528] v31'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605e43e6b40 con 0x5605e4373400 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 20 remove_watcher() i=7 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605e4373400 msgr2=0x5605e42b3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605e4373400 0x5605e42b3180 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605e3fc9c00 msgr2=0x5605e42b1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605e3fc9c00 0x5605e42b1b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605e43f6000 msgr2=0x5605e42b2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605e43f6000 0x5605e42b2c00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605e4373000 msgr2=0x5605e42b2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605e4373000 0x5605e42b2100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e4372c00 msgr2=0x5605e42b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e4372c00 0x5605e42b1600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5605e42ab890 tx=0x5605e40b9800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 msgr2=0x5605e42b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/19547312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc9800 0x5605e42b0b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5605e4e3a360 tx=0x5605e4312fc0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 reap_dead start 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed70b1700 1 -- 172.21.15.191:0/19547312 reap_dead start 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 shutdown_connections 2024-04-27T04:34:46.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 >> 172.21.15.191:0/19547312 conn(0x5605e3fc8c00 msgr2=0x5605e40b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 shutdown_connections 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/19547312 wait complete. 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e43f7c00 msgr2=0x5605e42b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e43f7c00 0x5605e42b3700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5605e437ff20 tx=0x5605e44252c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 msgr2=0x5605e42b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x5605e44067e0 tx=0x5605e40b98c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 shutdown_connections 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e43f7c00 0x5605e42b3700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/2612770767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e43ffc00 0x5605e42b2680 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.477+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 >> 172.21.15.191:0/2612770767 conn(0x5605e43ff800 msgr2=0x5605e40b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 shutdown_connections 2024-04-27T04:34:46.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/2612770767 wait complete. 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e3fc9400 msgr2=0x5605e42b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e3fc9400 0x5605e42b1080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5605e4e3aae0 tx=0x5605e4312de0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 msgr2=0x5605e42b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5605e41396b0 tx=0x5605e40b9a40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 shutdown_connections 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605e3fc9400 0x5605e42b1080 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 --2- 172.21.15.191:0/1910361667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605e3fc8800 0x5605e42b0580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 >> 172.21.15.191:0/1910361667 conn(0x5605e3fc8400 msgr2=0x5605e40b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 shutdown_connections 2024-04-27T04:34:46.527 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.478+0000 7f5ed9908a80 1 -- 172.21.15.191:0/1910361667 wait complete. 2024-04-27T04:34:46.528 INFO:teuthology.orchestra.run.smithi191.stdout:[ 2024-04-27T04:34:46.528 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:46.528 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:34:46.528 INFO:teuthology.orchestra.run.smithi191.stdout: "num_shards": 11, 2024-04-27T04:34:46.528 INFO:teuthology.orchestra.run.smithi191.stdout: "tenant": "", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "zonegroup": "29d2c105-c074-4639-b257-47b1060771f9", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_rule": "default-placement", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "explicit_placement": { 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "" 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "marker": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": "Normal", 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "versioned": false, 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "versioning_enabled": false, 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "object_lock_enabled": false, 2024-04-27T04:34:46.529 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_enabled": false, 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "mtime": "2024-04-27T04:34:45.736480Z", 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "creation_time": "2024-04-27T04:34:45.731629Z", 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "usage": {}, 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:46.530 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:46.531 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:46.531 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:46.531 INFO:teuthology.orchestra.run.smithi191.stdout:] 2024-04-27T04:34:46.531 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:34:45.736480Z', 'creation_time': '2024-04-27T04:34:45.731629Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-27T04:34:46.537 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T04:34:46.537 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T04:34:46.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T04:34:46.538 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T04:34:46.583 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:46.584 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ef8f6a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556883a44c00 0x556883d2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ef8f6a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556883d6f500 con 0x556883a44800 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ef8f6a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x556883ae6a00 con 0x556883a44c00 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ed09f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556883a44c00 0x556883d2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ed8a0700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x556883a44800 0x556883b30800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60356/0 (socket says 172.21.15.191:60356) 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.582+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/1431840724 learned_addr learned my addr 172.21.15.191:0/1431840724 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ed09f700 1 -- 172.21.15.191:0/1431840724 >> v1:172.21.15.138:6789/0 conn(0x556883a44800 legacy=0x556883b30800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ed09f700 1 -- 172.21.15.191:0/1431840724 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6dd40 con 0x556883a44c00 2024-04-27T04:34:46.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ec89e700 1 -- 172.21.15.191:0/1431840724 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572702946 0 0) 0x556884812000 con 0x556883a44800 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/1431840724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44c00 0x556883d2c580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x556883d27890 tx=0x556883b35920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfa4601282e7311 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ec89e700 1 -- 172.21.15.191:0/1431840724 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd4a0 con 0x556883a44c00 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ec89e700 1 -- 172.21.15.191:0/1431840724 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883d6f800 con 0x556883a44c00 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ec89e700 1 -- 172.21.15.191:0/1431840724 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd600 con 0x556883a44c00 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/1431840724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44c00 msgr2=0x556883d2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/1431840724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44c00 0x556883d2c580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x556883d27890 tx=0x556883b35920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/1431840724 shutdown_connections 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/1431840724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44c00 0x556883d2c580 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/1431840724 >> 172.21.15.191:0/1431840724 conn(0x556883a44400 msgr2=0x556883b30000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/1431840724 shutdown_connections 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.583+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/1431840724 wait complete. 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ef8f6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ef8f6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556883ae6500 con 0x556883a44800 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ed8a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ed8a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46958/0 (socket says 172.21.15.191:46958) 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.584+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/2220361575 learned_addr learned my addr 172.21.15.191:0/2220361575 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/2220361575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6dba0 con 0x556883a44800 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/2220361575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x556884815770 tx=0x556883b35b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae85f4e0ff0c363 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ec09d700 1 -- 172.21.15.191:0/2220361575 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd1e0 con 0x556883a44800 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ec09d700 1 -- 172.21.15.191:0/2220361575 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883d6fb00 con 0x556883a44800 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ec09d700 1 -- 172.21.15.191:0/2220361575 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd4a0 con 0x556883a44800 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/2220361575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 msgr2=0x556883d2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/2220361575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x556884815770 tx=0x556883b35b60 comp rx=0 tx=0).stop 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/2220361575 shutdown_connections 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/2220361575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/2220361575 >> 172.21.15.191:0/2220361575 conn(0x556883a44400 msgr2=0x556883b31000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/2220361575 shutdown_connections 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.585+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/2220361575 wait complete. 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ef8f6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ef8f6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556883ae63c0 con 0x556883a44800 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ed8a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ed8a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46970/0 (socket says 172.21.15.191:46970) 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/3626824852 learned_addr learned my addr 172.21.15.191:0/3626824852 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/3626824852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6da00 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x556884815e30 tx=0x556883b35a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.586+0000 7fd6ea89a700 1 -- 172.21.15.191:0/3626824852 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd600 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ea89a700 1 -- 172.21.15.191:0/3626824852 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883ae5c80 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556883a6d860 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ea89a700 1 -- 172.21.15.191:0/3626824852 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd1e0 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556883a6d6c0 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ef8f6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ef8f6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556883ae6000 con 0x556883a45400 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ea89a700 1 -- 172.21.15.191:0/3626824852 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5568847e6b00 con 0x556883a44800 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ee0a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ee0a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46978/0 (socket says 172.21.15.191:46978) 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/3759660922 learned_addr learned my addr 172.21.15.191:0/3759660922 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.587+0000 7fd6ea89a700 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0000 0x556883d2d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ea89a700 1 -- 172.21.15.191:0/3626824852 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x556883abf180 con 0x556883a44800 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/3759660922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6c000 con 0x556883a45400 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0000 0x556883d2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/3759660922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5568848ddb60 tx=0x556883d98de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f556287801b9c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6e8095700 1 -- 172.21.15.191:0/3759660922 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd600 con 0x556883a45400 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6e8095700 1 -- 172.21.15.191:0/3759660922 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883d6e300 con 0x556883a45400 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0000 0x556883d2d080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5568848dcf00 tx=0x556883d98240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6e8095700 1 -- 172.21.15.191:0/3759660922 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd1e0 con 0x556883a45400 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3759660922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 msgr2=0x556883d2cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3759660922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5568848ddb60 tx=0x556883d98de0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3759660922 shutdown_connections 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3759660922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3759660922 >> 172.21.15.191:0/3759660922 conn(0x556883a44c00 msgr2=0x556883b31800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3759660922 shutdown_connections 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.588+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3759660922 wait complete. 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ef8f6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ef8f6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556883a80640 con 0x556883a45400 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ee0a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ee0a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46984/0 (socket says 172.21.15.191:46984) 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 learned_addr learned my addr 172.21.15.191:0/96786489 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6cea0 con 0x556883a45400 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x556884815f80 tx=0x556883d98840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6e6892700 1 -- 172.21.15.191:0/96786489 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afda20 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6e6892700 1 -- 172.21.15.191:0/96786489 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883ae5c80 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556883a6c820 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.589+0000 7fd6e6892700 1 -- 172.21.15.191:0/96786489 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd600 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.590+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556883a6c9c0 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.590+0000 7fd6e6892700 1 -- 172.21.15.191:0/96786489 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5568847e6dc0 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.590+0000 7fd6e6892700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0c00 0x556883d2d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.590+0000 7fd6e6892700 1 -- 172.21.15.191:0/96786489 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x556883abf340 con 0x556883a45400 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.590+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0c00 0x556883d2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.590+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556883a45800 0x556883d2db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883a45c00 con 0x556883a45800 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556883a45800 0x556883d2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0c00 0x556883d2d600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556883d27890 tx=0x556883b35800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556883a45800 0x556883d2db80 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 20 realm 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1000 con 0x556883a45800 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.591+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1400 con 0x556883a45800 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1800 con 0x556883a45800 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556883de1c00 0x556883d2e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1000 con 0x556883de1c00 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556883de1c00 0x556883d2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.592+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556883de1c00 0x556883d2e100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ef8f6a80 20 searching for the correct realm 2024-04-27T04:34:46.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1400 con 0x556883de1c00 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ed8a0700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556883de1400 0x556883d2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1000 con 0x556883de1400 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556883de1400 0x556883d2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.593+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556883de1400 0x556883d2ec00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556883de1000 0x556883d2f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556883de1000 0x556883d2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556883de1000 0x556883d2f180 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.594+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.595+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1400 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.596+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1400 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.597+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.598+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1400 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.599+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1400 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.600+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1c00 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1400 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883a45c00 con 0x556883a45800 2024-04-27T04:34:46.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.601+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c400 con 0x556883a45800 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c800 con 0x556883de1c00 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 zone default found 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 4 Realm: () 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:46.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 zonegroup default 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9cc00 con 0x556883de1000 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d000 con 0x556883de1c00 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.602+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d400 con 0x556883de1c00 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.603+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.603+0000 7fd6ef8f6a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.603+0000 7fd6ef8f6a80 20 started sync module instance, tier type = 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.603+0000 7fd6ef8f6a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.603+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d800 con 0x556883de1400 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.604+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.604+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93907376552704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c800 con 0x556883de1400 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.604+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d000 con 0x556883a45800 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.605+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93907376552704] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.605+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.605+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93907376553856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d400 con 0x556883a45800 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.605+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c400 con 0x556883de1c00 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.606+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93907376553856] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.606+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.606+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93907376555008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9cc00 con 0x556883de1c00 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.606+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883a45c00 con 0x556883de1400 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.607+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93907376555008] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.607+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.607+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93907376556160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883de1800 con 0x556883de1400 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.607+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883a45800 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.608+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.608+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93907376556160] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.608+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93907376557312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d800 con 0x556883a45800 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.608+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c800 con 0x556883de1000 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.609+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.609+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93907376558464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9dc00 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.609+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d000 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.609+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93907376557312] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.610+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93907376558464] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.610+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.610+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93907376559616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d400 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.610+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c000 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93907376559616] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93907376560768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d800 con 0x556883de1000 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=0 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=1 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=2 2024-04-27T04:34:46.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=3 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=4 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=5 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ef8f6a80 20 add_watcher() i=6 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.611+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93907376560768] v31'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.612+0000 7fd6ef8f6a80 20 add_watcher() i=7 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.612+0000 7fd6ef8f6a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.612+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c800 con 0x556883a45800 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.612+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.612+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93907378126848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9dc00 con 0x556883a45800 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93907378126848] v31'106 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6d486e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ef8f6a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ef8f6a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ef8f6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.613+0000 7fd6ef8f6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556883a80780 con 0x556883e9d400 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6ed09f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6ed09f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46988/0 (socket says 172.21.15.191:46988) 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6ed09f700 1 -- 172.21.15.191:0/3637322766 learned_addr learned my addr 172.21.15.191:0/3637322766 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6ed09f700 1 -- 172.21.15.191:0/3637322766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6cb60 con 0x556883e9d400 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/3637322766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x556883e3b7a0 tx=0x556883b35980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5b71d909b4b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6d406d700 1 -- 172.21.15.191:0/3637322766 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883afd760 con 0x556883e9d400 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6d406d700 1 -- 172.21.15.191:0/3637322766 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883eb4000 con 0x556883e9d400 2024-04-27T04:34:46.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.614+0000 7fd6d406d700 1 -- 172.21.15.191:0/3637322766 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883eb8000 con 0x556883e9d400 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3637322766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 msgr2=0x556883d2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3637322766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x556883e3b7a0 tx=0x556883b35980 comp rx=0 tx=0).stop 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3637322766 shutdown_connections 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3637322766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3637322766 >> 172.21.15.191:0/3637322766 conn(0x556883e9d000 msgr2=0x556883b32000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3637322766 shutdown_connections 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3637322766 wait complete. 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 Processor -- start 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- start start 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.615+0000 7fd6ef8f6a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556883ae6000 con 0x556883e9d400 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ed09f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ed09f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47000/0 (socket says 172.21.15.191:47000) 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ed09f700 1 -- 172.21.15.191:0/947199848 learned_addr learned my addr 172.21.15.191:0/947199848 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ed09f700 1 -- 172.21.15.191:0/947199848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556883a6cd00 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ed09f700 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x556883eaf260 tx=0x556883b358c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6d286a700 1 -- 172.21.15.191:0/947199848 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883eb8160 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6d286a700 1 -- 172.21.15.191:0/947199848 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556883eb4180 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556883a6c340 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6d286a700 1 -- 172.21.15.191:0/947199848 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556883eb82c0 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.616+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556883a6dd40 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.617+0000 7fd6d286a700 1 -- 172.21.15.191:0/947199848 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5568847e7b80 con 0x556883e9d400 2024-04-27T04:34:46.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.617+0000 7fd6d286a700 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883f36000 0x556883d2f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.617+0000 7fd6d286a700 1 -- 172.21.15.191:0/947199848 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x556883abf500 con 0x556883e9d400 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.617+0000 7fd6ef8f6a80 5 note: GC not initialized 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.617+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883f36000 0x556883d2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.617+0000 7fd6ee0a1700 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883f36000 0x556883d2f700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x556883e3bef0 tx=0x556883eb9320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6d0065700 20 reqs_thread_entry: start 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6ef8f6a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6ef8f6a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6ce061700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:16 2024 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6cd83f700 20 reqs_thread_entry: start 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.618+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9d800 con 0x556883de1400 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c800 con 0x556883de1000 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36400 con 0x556883de1400 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.619+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36800 con 0x556883de1000 2024-04-27T04:34:46.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(65 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 4.25 4:a741c040:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36c00 con 0x556883de1c00 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 4.39 4:9e5e97cb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37000 con 0x556883de1c00 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 4.1c 4:381df985:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37400 con 0x556883de1000 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 4.1c 4:3a1291c0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37800 con 0x556883de1000 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 4.5 4:a135d554:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37c00 con 0x556883de1c00 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 4.2a 4:57f23c0a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36c00 con 0x556883a45800 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37000 con 0x556883a45800 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 4.11 4:8b5d38c9:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c400 con 0x556883de1000 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:74 4.29 4:95afba01:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9cc00 con 0x556883de1c00 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.2e 4:74385e64:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36800 con 0x556883de1c00 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 4.1f 4:fa83b5f2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37400 con 0x556883de1c00 2024-04-27T04:34:46.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.620+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(76 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6d486e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.621+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93907378126848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37800 con 0x556883a45800 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.622+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 93907378126848] v31'107 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.622+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93907376552704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36c00 con 0x556883de1400 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.623+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 93907376552704] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.623+0000 7fd6ef8f6a80 20 remove_watcher() i=0 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.623+0000 7fd6ef8f6a80 2 removed watcher, disabling cache 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.623+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93907376553856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37000 con 0x556883a45800 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.624+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 93907376553856] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.624+0000 7fd6ef8f6a80 20 remove_watcher() i=1 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.624+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93907376555008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9dc00 con 0x556883de1c00 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.626+0000 7fd6ed8a0700 1 -- 172.21.15.191:0/96786489 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 93907376555008] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d200 con 0x556883de1c00 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.626+0000 7fd6ef8f6a80 20 remove_watcher() i=2 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.626+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93907376556160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883e9c800 con 0x556883de1400 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.627+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 93907376556160] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8d440 con 0x556883de1400 2024-04-27T04:34:46.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.627+0000 7fd6ef8f6a80 20 remove_watcher() i=3 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.627+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93907376557312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37c00 con 0x556883a45800 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.628+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 93907376557312] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883a45800 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.628+0000 7fd6ef8f6a80 20 remove_watcher() i=4 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.628+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93907376558464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36800 con 0x556883de1000 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.629+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 93907376558464] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.629+0000 7fd6ef8f6a80 20 remove_watcher() i=5 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.629+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93907376559616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f37400 con 0x556883de1000 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.630+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 93907376559616] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.630+0000 7fd6ef8f6a80 20 remove_watcher() i=6 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.630+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93907376560768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556883f36400 con 0x556883de1000 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.630+0000 7fd6ee0a1700 1 -- 172.21.15.191:0/96786489 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 93907376560768] v31'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556883e8cb40 con 0x556883de1000 2024-04-27T04:34:46.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.630+0000 7fd6ef8f6a80 20 remove_watcher() i=7 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556883de1000 msgr2=0x556883d2f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556883de1000 0x556883d2f180 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556883a45800 msgr2=0x556883d2db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556883a45800 0x556883d2db80 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556883de1400 msgr2=0x556883d2ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556883de1400 0x556883d2ec00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556883de1c00 msgr2=0x556883d2e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556883de1c00 0x556883d2e100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0c00 msgr2=0x556883d2d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0c00 0x556883d2d600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556883d27890 tx=0x556883b35800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 msgr2=0x556883d2cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/96786489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a45400 0x556883d2cb00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x556884815f80 tx=0x556883d98840 comp rx=0 tx=0).stop 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 reap_dead start 2024-04-27T04:34:46.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ed09f700 1 -- 172.21.15.191:0/96786489 reap_dead start 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 shutdown_connections 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 >> 172.21.15.191:0/96786489 conn(0x556883a44c00 msgr2=0x556883b31800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 shutdown_connections 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/96786489 wait complete. 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883f36000 msgr2=0x556883d2f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883f36000 0x556883d2f700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x556883e3bef0 tx=0x556883eb9320 comp rx=0 tx=0).stop 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 msgr2=0x556883d2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x556883eaf260 tx=0x556883b358c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 shutdown_connections 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883f36000 0x556883d2f700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/947199848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883e9d400 0x556883d2e680 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 >> 172.21.15.191:0/947199848 conn(0x556883e9d000 msgr2=0x556883b32000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 shutdown_connections 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.631+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/947199848 wait complete. 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0000 msgr2=0x556883d2d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0000 0x556883d2d080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5568848dcf00 tx=0x556883d98240 comp rx=0 tx=0).stop 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 msgr2=0x556883d2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x556884815e30 tx=0x556883b35a40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 shutdown_connections 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556883de0000 0x556883d2d080 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 --2- 172.21.15.191:0/3626824852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556883a44800 0x556883d2c580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 >> 172.21.15.191:0/3626824852 conn(0x556883a44400 msgr2=0x556883b31000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 shutdown_connections 2024-04-27T04:34:46.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.632+0000 7fd6ef8f6a80 1 -- 172.21.15.191:0/3626824852 wait complete. 2024-04-27T04:34:46.656 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:46.656 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:34:46.656 INFO:teuthology.orchestra.run.smithi191.stdout: "num_shards": 11, 2024-04-27T04:34:46.656 INFO:teuthology.orchestra.run.smithi191.stdout: "tenant": "", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "zonegroup": "29d2c105-c074-4639-b257-47b1060771f9", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_rule": "default-placement", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "explicit_placement": { 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "" 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "marker": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": "Normal", 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "versioned": false, 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "versioning_enabled": false, 2024-04-27T04:34:46.657 INFO:teuthology.orchestra.run.smithi191.stdout: "object_lock_enabled": false, 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_enabled": false, 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1", 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "mtime": "2024-04-27T04:34:45.736480Z", 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "creation_time": "2024-04-27T04:34:45.731629Z", 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "usage": { 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "rgw.main": { 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "size": 3, 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "size_actual": 4096, 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "size_utilized": 3, 2024-04-27T04:34:46.658 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb": 1, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb_actual": 4, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb_utilized": 1, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "num_objects": 1 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:46.659 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:46.660 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:34:45.736480Z', 'creation_time': '2024-04-27T04:34:45.731629Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T04:34:46.660 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-27T04:34:46.660 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-27T04:34:46.660 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-04-27T04:34:46.706 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:46.706 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.704+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.704+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a87eda80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a87eda80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56118e315500 con 0x56118dfeac00 2024-04-27T04:34:46.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a87eda80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x56118e08ca00 con 0x56118dfea800 2024-04-27T04:34:46.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a6797700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a5f96700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x56118dfeac00 0x56118e0d6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60358/0 (socket says 172.21.15.191:60358) 2024-04-27T04:34:46.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a5f96700 1 -- 172.21.15.191:0/2810478460 learned_addr learned my addr 172.21.15.191:0/2810478460 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:46.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a6797700 1 -- 172.21.15.191:0/2810478460 >> v1:172.21.15.138:6789/0 conn(0x56118dfeac00 legacy=0x56118e0d6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a6797700 1 -- 172.21.15.191:0/2810478460 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e013d40 con 0x56118dfea800 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.705+0000 7fa9a6797700 1 --2- 172.21.15.191:0/2810478460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x56118e2cd9e0 tx=0x56118e0db920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdd89a8f57c0795 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a5795700 1 -- 172.21.15.191:0/2810478460 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a34a0 con 0x56118dfea800 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a5795700 1 -- 172.21.15.191:0/2810478460 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e315800 con 0x56118dfea800 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a5795700 1 -- 172.21.15.191:0/2810478460 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a3600 con 0x56118dfea800 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2810478460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 msgr2=0x56118e2d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2810478460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x56118e2cd9e0 tx=0x56118e0db920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2810478460 shutdown_connections 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2810478460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2810478460 >> 172.21.15.191:0/2810478460 conn(0x56118dfea400 msgr2=0x56118e0d6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2810478460 shutdown_connections 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.706+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2810478460 wait complete. 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a87eda80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a87eda80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56118e08c500 con 0x56118dfea800 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a6797700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a6797700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47010/0 (socket says 172.21.15.191:47010) 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.707+0000 7fa9a6797700 1 -- 172.21.15.191:0/2198346462 learned_addr learned my addr 172.21.15.191:0/2198346462 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a6797700 1 -- 172.21.15.191:0/2198346462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e013ba0 con 0x56118dfea800 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a6797700 1 --2- 172.21.15.191:0/2198346462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56118edc3740 tx=0x56118e0db800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d809523c37ca2914 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a4f94700 1 -- 172.21.15.191:0/2198346462 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a3760 con 0x56118dfea800 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a4f94700 1 -- 172.21.15.191:0/2198346462 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e315b00 con 0x56118dfea800 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a4f94700 1 -- 172.21.15.191:0/2198346462 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a31e0 con 0x56118dfea800 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2198346462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 msgr2=0x56118e2d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2198346462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56118edc3740 tx=0x56118e0db800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2198346462 shutdown_connections 2024-04-27T04:34:46.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2198346462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2198346462 >> 172.21.15.191:0/2198346462 conn(0x56118dfea400 msgr2=0x56118e0d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2198346462 shutdown_connections 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.708+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2198346462 wait complete. 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a87eda80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a87eda80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56118e08c3c0 con 0x56118dfea800 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a6797700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a6797700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47020/0 (socket says 172.21.15.191:47020) 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a6797700 1 -- 172.21.15.191:0/2442433782 learned_addr learned my addr 172.21.15.191:0/2442433782 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a6797700 1 -- 172.21.15.191:0/2442433782 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e013a00 con 0x56118dfea800 2024-04-27T04:34:46.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.709+0000 7fa9a6797700 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x56118edc35f0 tx=0x56118e0db920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a3791700 1 -- 172.21.15.191:0/2442433782 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a34a0 con 0x56118dfea800 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a3791700 1 -- 172.21.15.191:0/2442433782 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e315c80 con 0x56118dfea800 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56118e013860 con 0x56118dfea800 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a3791700 1 -- 172.21.15.191:0/2442433782 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a3600 con 0x56118dfea800 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56118e0136c0 con 0x56118dfea800 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a3791700 1 -- 172.21.15.191:0/2442433782 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56118ed8cb00 con 0x56118dfea800 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a87eda80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a87eda80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56118e08c000 con 0x56118dfeb400 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a3791700 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37a000 0x56118e2d3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.710+0000 7fa9a3791700 1 -- 172.21.15.191:0/2442433782 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56118e065180 con 0x56118dfea800 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6797700 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37a000 0x56118e2d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6f98700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6f98700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47022/0 (socket says 172.21.15.191:47022) 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6f98700 1 -- 172.21.15.191:0/3267026115 learned_addr learned my addr 172.21.15.191:0/3267026115 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6f98700 1 -- 172.21.15.191:0/3267026115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e012000 con 0x56118dfeb400 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6797700 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37a000 0x56118e2d3080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56118ee7ced0 tx=0x56118e33eb40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/3267026115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x56118ee7c210 tx=0x56118e33ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5972f74fafcacf7a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a0f8c700 1 -- 172.21.15.191:0/3267026115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a34a0 con 0x56118dfeb400 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a0f8c700 1 -- 172.21.15.191:0/3267026115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e08bc80 con 0x56118dfeb400 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a0f8c700 1 -- 172.21.15.191:0/3267026115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a3600 con 0x56118dfeb400 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3267026115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 msgr2=0x56118e2d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.711+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/3267026115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x56118ee7c210 tx=0x56118e33ee40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.720 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3267026115 shutdown_connections 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/3267026115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3267026115 >> 172.21.15.191:0/3267026115 conn(0x56118dfeac00 msgr2=0x56118e0d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3267026115 shutdown_connections 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3267026115 wait complete. 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a87eda80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56118e026640 con 0x56118dfeb400 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a6f98700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a6f98700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47026/0 (socket says 172.21.15.191:47026) 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 learned_addr learned my addr 172.21.15.191:0/4101871674 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e012ea0 con 0x56118dfeb400 2024-04-27T04:34:46.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.712+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56118ee7c810 tx=0x56118e33ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.713+0000 7fa99f789700 1 -- 172.21.15.191:0/4101871674 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a3760 con 0x56118dfeb400 2024-04-27T04:34:46.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.713+0000 7fa99f789700 1 -- 172.21.15.191:0/4101871674 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e08bb00 con 0x56118dfeb400 2024-04-27T04:34:46.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.713+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56118e012820 con 0x56118dfeb400 2024-04-27T04:34:46.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.713+0000 7fa99f789700 1 -- 172.21.15.191:0/4101871674 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e0a34a0 con 0x56118dfeb400 2024-04-27T04:34:46.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.713+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56118e0129c0 con 0x56118dfeb400 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.713+0000 7fa99f789700 1 -- 172.21.15.191:0/4101871674 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56118ed8cdc0 con 0x56118dfeb400 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa99f789700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37ac00 0x56118e2d3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa99f789700 1 -- 172.21.15.191:0/4101871674 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56118e065340 con 0x56118dfeb400 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37ac00 0x56118e2d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56118dfeb800 0x56118e2d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118dfebc00 con 0x56118dfeb800 2024-04-27T04:34:46.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56118dfeb800 0x56118e2d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37ac00 0x56118e2d3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56118e379aa0 tx=0x56118e4280c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.714+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56118dfeb800 0x56118e2d3b80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 realm 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b000 con 0x56118dfeb800 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b400 con 0x56118dfeb800 2024-04-27T04:34:46.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b800 con 0x56118dfeb800 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56118e37bc00 0x56118e2d4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b000 con 0x56118e37bc00 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.715+0000 7fa9a6797700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56118e37bc00 0x56118e2d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a6797700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56118e37bc00 0x56118e2d4100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a87eda80 20 searching for the correct realm 2024-04-27T04:34:46.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b400 con 0x56118e37bc00 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a6797700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56118e37b400 0x56118e2d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b000 con 0x56118e37b400 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56118e37b400 0x56118e2d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.716+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56118e37b400 0x56118e2d4c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56118e37b000 0x56118e2d5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56118e37b000 0x56118e2d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56118e37b000 0x56118e2d5180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.717+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.718+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b400 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.719+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b400 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.720+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.721+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b400 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.722+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b400 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37bc00 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.723+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b400 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118dfebc00 con 0x56118dfeb800 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c400 con 0x56118dfeb800 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.724+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c800 con 0x56118e37bc00 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 zone default found 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 4 Realm: () 2024-04-27T04:34:46.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 zonegroup default 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43cc00 con 0x56118e37b000 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d000 con 0x56118e37bc00 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d400 con 0x56118e37bc00 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.725+0000 7fa9a87eda80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.726+0000 7fa9a87eda80 20 started sync module instance, tier type = 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.726+0000 7fa9a87eda80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.726+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d800 con 0x56118e37b400 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.727+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.727+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94633399720704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c800 con 0x56118e37b400 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.727+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d000 con 0x56118dfeb800 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.728+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94633399720704] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.728+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.728+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94633399721856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d400 con 0x56118dfeb800 2024-04-27T04:34:46.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.728+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c400 con 0x56118e37bc00 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.729+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94633399721856] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.729+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.729+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94633399723008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43cc00 con 0x56118e37bc00 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.729+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118dfebc00 con 0x56118e37b400 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.730+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.730+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94633399723008] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.730+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94633399724160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e37b800 con 0x56118e37b400 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.730+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118dfeb800 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.731+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.731+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94633399725312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d800 con 0x56118dfeb800 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.731+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c800 con 0x56118e37b000 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.731+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94633399724160] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.732+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.732+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94633399726464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43dc00 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.732+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94633399725312] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.732+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d000 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.733+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94633399726464] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.733+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.733+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94633399727616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d400 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.733+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c000 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94633399727616] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94633399728768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d800 con 0x56118e37b000 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=0 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=1 2024-04-27T04:34:46.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=2 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=3 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=4 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=5 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.734+0000 7fa9a87eda80 20 add_watcher() i=6 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.735+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94633399728768] v31'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.735+0000 7fa9a87eda80 20 add_watcher() i=7 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.735+0000 7fa9a87eda80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.735+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c800 con 0x56118dfeb800 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.735+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.735+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94633401286656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43dc00 con 0x56118dfeb800 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.736+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94633401286656] v31'108 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.736+0000 7fa98d765700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.736+0000 7fa9a87eda80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:46.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.736+0000 7fa9a87eda80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:46.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.736+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.736+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a87eda80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a87eda80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56118e026780 con 0x56118e43d400 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a5f96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a5f96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47028/0 (socket says 172.21.15.191:47028) 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a5f96700 1 -- 172.21.15.191:0/1221132934 learned_addr learned my addr 172.21.15.191:0/1221132934 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a5f96700 1 -- 172.21.15.191:0/1221132934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e012b60 con 0x56118e43d400 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.737+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/1221132934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56118e3df5f0 tx=0x56118e33ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60da8d09555a0b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa98cf64700 1 -- 172.21.15.191:0/1221132934 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e45a000 con 0x56118e43d400 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa98cf64700 1 -- 172.21.15.191:0/1221132934 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e45e000 con 0x56118e43d400 2024-04-27T04:34:46.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa98cf64700 1 -- 172.21.15.191:0/1221132934 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e45a160 con 0x56118e43d400 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 -- 172.21.15.191:0/1221132934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 msgr2=0x56118e2d4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/1221132934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56118e3df5f0 tx=0x56118e33ff80 comp rx=0 tx=0).stop 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 -- 172.21.15.191:0/1221132934 shutdown_connections 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/1221132934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 -- 172.21.15.191:0/1221132934 >> 172.21.15.191:0/1221132934 conn(0x56118e43d000 msgr2=0x56118e0d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 -- 172.21.15.191:0/1221132934 shutdown_connections 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.738+0000 7fa9a87eda80 1 -- 172.21.15.191:0/1221132934 wait complete. 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a87eda80 1 Processor -- start 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a87eda80 1 -- start start 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a87eda80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a87eda80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56118e08c000 con 0x56118e43d400 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a5f96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a5f96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47042/0 (socket says 172.21.15.191:47042) 2024-04-27T04:34:46.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a5f96700 1 -- 172.21.15.191:0/3230683673 learned_addr learned my addr 172.21.15.191:0/3230683673 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a5f96700 1 -- 172.21.15.191:0/3230683673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118e012d00 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a5f96700 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x56118e45c210 tx=0x56118e4284e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa98b761700 1 -- 172.21.15.191:0/3230683673 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e45a2c0 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa98b761700 1 -- 172.21.15.191:0/3230683673 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56118e45e180 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56118e012340 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa98b761700 1 -- 172.21.15.191:0/3230683673 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56118e45a000 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.739+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56118e012b60 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.740+0000 7fa98b761700 1 -- 172.21.15.191:0/3230683673 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56118ed8db80 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.740+0000 7fa98b761700 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e4de000 0x56118e2d5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.740+0000 7fa98b761700 1 -- 172.21.15.191:0/3230683673 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x56118e065500 con 0x56118e43d400 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.740+0000 7fa9a87eda80 5 note: GC not initialized 2024-04-27T04:34:46.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.740+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e4de000 0x56118e2d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.741+0000 7fa9a6f98700 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e4de000 0x56118e2d5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x56118e3dfec0 tx=0x56118e428360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.741+0000 7fa988f5c700 20 reqs_thread_entry: start 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.741+0000 7fa9a87eda80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.741+0000 7fa9a87eda80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.741+0000 7fa986f58700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.741+0000 7fa986736700 20 reqs_thread_entry: start 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d800 con 0x56118dfeb800 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c800 con 0x56118e37b400 2024-04-27T04:34:46.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.742+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4de400 con 0x56118e37b000 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 4.25 4:a741c040:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4de800 con 0x56118e37bc00 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 4.39 4:9e5e97cb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4dec00 con 0x56118e37bc00 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 4.1c 4:381df985:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df000 con 0x56118e37b000 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 4.1c 4:3a1291c0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df400 con 0x56118e37b000 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 4.5 4:a135d554:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df800 con 0x56118e37bc00 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 4.2a 4:57f23c0a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4dfc00 con 0x56118dfeb800 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4de800 con 0x56118dfeb800 2024-04-27T04:34:46.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 4.11 4:8b5d38c9:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4dec00 con 0x56118e37b000 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 4.29 4:95afba01:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c400 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:74 4.2e 4:74385e64:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43cc00 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.1f 4:fa83b5f2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4de400 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.743+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.744+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.744+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.744+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.744+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df000 con 0x56118e37b400 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.745+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(76 foo.buckets [call] v31'15 uv15 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.745+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df400 con 0x56118e37b400 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(77 foo.buckets [call] v31'16 uv16 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a87eda80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4dfc00 con 0x56118dfeb800 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(78 foo [stat out=16b,read 0~357 out=357b] v0'0 uv11 ondisk = 0) v8 ==== 189+0+373 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a87eda80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4de800 con 0x56118e37b400 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.746+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) v8 ==== 155+0+58 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.747+0000 7fa98d765700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.747+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94633401286656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43dc00 con 0x56118dfeb800 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.748+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 94633401286656] v31'109 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.748+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94633399720704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43d800 con 0x56118e37b400 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.749+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 94633399720704] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.749+0000 7fa9a87eda80 20 remove_watcher() i=0 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.749+0000 7fa9a87eda80 2 removed watcher, disabling cache 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.749+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94633399721856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df000 con 0x56118dfeb800 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.750+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 94633399721856] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.750+0000 7fa9a87eda80 20 remove_watcher() i=1 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.750+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94633399723008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df400 con 0x56118e37bc00 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.751+0000 7fa9a6797700 1 -- 172.21.15.191:0/4101871674 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 94633399723008] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439200 con 0x56118e37bc00 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.751+0000 7fa9a87eda80 20 remove_watcher() i=2 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.751+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94633399724160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e43c800 con 0x56118e37b400 2024-04-27T04:34:46.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.752+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 94633399724160] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e439440 con 0x56118e37b400 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.752+0000 7fa9a87eda80 20 remove_watcher() i=3 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.752+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94633399725312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118dfebc00 con 0x56118dfeb800 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.753+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 94633399725312] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118dfeb800 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.753+0000 7fa9a87eda80 20 remove_watcher() i=4 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.753+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94633399726464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df800 con 0x56118e37b000 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.754+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 94633399726464] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.754+0000 7fa9a87eda80 20 remove_watcher() i=5 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.754+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94633399727616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4de400 con 0x56118e37b000 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.755+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 94633399727616] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.755+0000 7fa9a87eda80 20 remove_watcher() i=6 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.755+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94633399728768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56118e4df400 con 0x56118e37b000 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a6f98700 1 -- 172.21.15.191:0/4101871674 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 94633399728768] v31'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56118e438b40 con 0x56118e37b000 2024-04-27T04:34:46.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 20 remove_watcher() i=7 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56118e37b000 msgr2=0x56118e2d5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56118e37b000 0x56118e2d5180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56118dfeb800 msgr2=0x56118e2d3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56118dfeb800 0x56118e2d3b80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56118e37b400 msgr2=0x56118e2d4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56118e37b400 0x56118e2d4c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56118e37bc00 msgr2=0x56118e2d4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56118e37bc00 0x56118e2d4100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37ac00 msgr2=0x56118e2d3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37ac00 0x56118e2d3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56118e379aa0 tx=0x56118e4280c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 msgr2=0x56118e2d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/4101871674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfeb400 0x56118e2d2b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56118ee7c810 tx=0x56118e33ea20 comp rx=0 tx=0).stop 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 reap_dead start 2024-04-27T04:34:46.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a5f96700 1 -- 172.21.15.191:0/4101871674 reap_dead start 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 shutdown_connections 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 >> 172.21.15.191:0/4101871674 conn(0x56118dfeac00 msgr2=0x56118e0d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 shutdown_connections 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.756+0000 7fa9a87eda80 1 -- 172.21.15.191:0/4101871674 wait complete. 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e4de000 msgr2=0x56118e2d5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e4de000 0x56118e2d5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x56118e3dfec0 tx=0x56118e428360 comp rx=0 tx=0).stop 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 msgr2=0x56118e2d4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x56118e45c210 tx=0x56118e4284e0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 shutdown_connections 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e4de000 0x56118e2d5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/3230683673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118e43d400 0x56118e2d4680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 >> 172.21.15.191:0/3230683673 conn(0x56118e43d000 msgr2=0x56118e0d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 shutdown_connections 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/3230683673 wait complete. 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37a000 msgr2=0x56118e2d3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37a000 0x56118e2d3080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56118ee7ced0 tx=0x56118e33eb40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 msgr2=0x56118e2d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x56118edc35f0 tx=0x56118e0db920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 shutdown_connections 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56118e37a000 0x56118e2d3080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 --2- 172.21.15.191:0/2442433782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56118dfea800 0x56118e2d2580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 >> 172.21.15.191:0/2442433782 conn(0x56118dfea400 msgr2=0x56118e0d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 shutdown_connections 2024-04-27T04:34:46.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.757+0000 7fa9a87eda80 1 -- 172.21.15.191:0/2442433782 wait complete. 2024-04-27T04:34:46.779 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:46.779 INFO:teuthology.orchestra.run.smithi191.stdout: "stats": { 2024-04-27T04:34:46.779 INFO:teuthology.orchestra.run.smithi191.stdout: "size": 3, 2024-04-27T04:34:46.779 INFO:teuthology.orchestra.run.smithi191.stdout: "size_actual": 4096, 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb": 1, 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb_actual": 4, 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: "num_objects": 1 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: "last_stats_sync": "2024-04-27T04:34:46.747033Z", 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: "last_stats_update": "2024-04-27T04:34:46.745986Z" 2024-04-27T04:34:46.780 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:46.780 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-04-27T04:34:46.747033Z', 'last_stats_update': '2024-04-27T04:34:46.745986Z'} 2024-04-27T04:34:46.785 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:46.785 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-27T04:34:46.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-27T04:34:46.785 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-04-27T04:34:46.832 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:46.833 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfd1cca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfd1cca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5605ae6fd500 con 0x5605ae3d2c00 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfd1cca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5605ae474a00 con 0x5605ae3d2800 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfb176700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfa975700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5605ae3d2c00 0x5605ae4be800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60372/0 (socket says 172.21.15.191:60372) 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfa975700 1 -- 172.21.15.191:0/176139111 learned_addr learned my addr 172.21.15.191:0/176139111 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfb176700 1 -- 172.21.15.191:0/176139111 >> v1:172.21.15.138:6789/0 conn(0x5605ae3d2c00 legacy=0x5605ae4be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.831+0000 7fbdfb176700 1 -- 172.21.15.191:0/176139111 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3fbd40 con 0x5605ae3d2800 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfb176700 1 --2- 172.21.15.191:0/176139111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b59e0 tx=0x5605ae4c3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b117a82d4be8a87f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfa174700 1 -- 172.21.15.191:0/176139111 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b4a0 con 0x5605ae3d2800 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfa174700 1 -- 172.21.15.191:0/176139111 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae6fd800 con 0x5605ae3d2800 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfa174700 1 -- 172.21.15.191:0/176139111 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b600 con 0x5605ae3d2800 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/176139111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 msgr2=0x5605ae6ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/176139111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b59e0 tx=0x5605ae4c3920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/176139111 shutdown_connections 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/176139111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/176139111 >> 172.21.15.191:0/176139111 conn(0x5605ae3d2400 msgr2=0x5605ae4be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/176139111 shutdown_connections 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.832+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/176139111 wait complete. 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfd1cca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfd1cca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ae474500 con 0x5605ae3d2800 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfb176700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfb176700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47060/0 (socket says 172.21.15.191:47060) 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfb176700 1 -- 172.21.15.191:0/4118559734 learned_addr learned my addr 172.21.15.191:0/4118559734 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.833+0000 7fbdfb176700 1 -- 172.21.15.191:0/4118559734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3fbba0 con 0x5605ae3d2800 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfb176700 1 --2- 172.21.15.191:0/4118559734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x5605af1ab8c0 tx=0x5605ae4c3800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a5364184867d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdf9973700 1 -- 172.21.15.191:0/4118559734 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b760 con 0x5605ae3d2800 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdf9973700 1 -- 172.21.15.191:0/4118559734 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae6fdb00 con 0x5605ae3d2800 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdf9973700 1 -- 172.21.15.191:0/4118559734 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b1e0 con 0x5605ae3d2800 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/4118559734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 msgr2=0x5605ae6ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/4118559734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x5605af1ab8c0 tx=0x5605ae4c3800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/4118559734 shutdown_connections 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/4118559734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/4118559734 >> 172.21.15.191:0/4118559734 conn(0x5605ae3d2400 msgr2=0x5605ae4bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/4118559734 shutdown_connections 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/4118559734 wait complete. 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.834+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfd1cca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfd1cca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ae4743c0 con 0x5605ae3d2800 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfb176700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfb176700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47074/0 (socket says 172.21.15.191:47074) 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfb176700 1 -- 172.21.15.191:0/3123473124 learned_addr learned my addr 172.21.15.191:0/3123473124 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfb176700 1 -- 172.21.15.191:0/3123473124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3fba00 con 0x5605ae3d2800 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfb176700 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b4a50 tx=0x5605ae4c3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdf8170700 1 -- 172.21.15.191:0/3123473124 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b4a0 con 0x5605ae3d2800 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdf8170700 1 -- 172.21.15.191:0/3123473124 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae6fdc80 con 0x5605ae3d2800 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdf8170700 1 -- 172.21.15.191:0/3123473124 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b600 con 0x5605ae3d2800 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605ae3fb860 con 0x5605ae3d2800 2024-04-27T04:34:46.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.835+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605ae3fb6c0 con 0x5605ae3d2800 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdf8170700 1 -- 172.21.15.191:0/3123473124 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5605af174b00 con 0x5605ae3d2800 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfd1cca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfd1cca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ae40e640 con 0x5605ae3d3400 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdf8170700 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75a000 0x5605ae6bb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdf8170700 1 -- 172.21.15.191:0/3123473124 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5605ae44d180 con 0x5605ae3d2800 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb977700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb977700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47084/0 (socket says 172.21.15.191:47084) 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb977700 1 -- 172.21.15.191:0/3988631464 learned_addr learned my addr 172.21.15.191:0/3988631464 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb176700 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75a000 0x5605ae6bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb977700 1 -- 172.21.15.191:0/3988631464 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3fa000 con 0x5605ae3d3400 2024-04-27T04:34:46.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb176700 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75a000 0x5605ae6bb080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5605af25eed0 tx=0x5605ae722c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.836+0000 7fbdfb977700 1 --2- 172.21.15.191:0/3988631464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b5860 tx=0x5605ae722b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b047d712b7d564db server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdf596b700 1 -- 172.21.15.191:0/3988631464 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b4a0 con 0x5605ae3d3400 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdf596b700 1 -- 172.21.15.191:0/3988631464 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae473c80 con 0x5605ae3d3400 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdf596b700 1 -- 172.21.15.191:0/3988631464 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b600 con 0x5605ae3d3400 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3988631464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 msgr2=0x5605ae6bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3988631464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b5860 tx=0x5605ae722b40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3988631464 shutdown_connections 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3988631464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3988631464 >> 172.21.15.191:0/3988631464 conn(0x5605ae3d2c00 msgr2=0x5605ae4bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3988631464 shutdown_connections 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3988631464 wait complete. 2024-04-27T04:34:46.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.837+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfd1cca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfd1cca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ae40e8c0 con 0x5605ae3d3400 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfb977700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfb977700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47086/0 (socket says 172.21.15.191:47086) 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 learned_addr learned my addr 172.21.15.191:0/2970575316 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3faea0 con 0x5605ae3d3400 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfb977700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b59b0 tx=0x5605ae7226c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdf4168700 1 -- 172.21.15.191:0/2970575316 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b760 con 0x5605ae3d3400 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdf4168700 1 -- 172.21.15.191:0/2970575316 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae473b00 con 0x5605ae3d3400 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdf4168700 1 -- 172.21.15.191:0/2970575316 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae48b4a0 con 0x5605ae3d3400 2024-04-27T04:34:46.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605ae3fa820 con 0x5605ae3d3400 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.838+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605ae3fa9c0 con 0x5605ae3d3400 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdf4168700 1 -- 172.21.15.191:0/2970575316 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5605af174dc0 con 0x5605ae3d3400 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdf4168700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75ac00 0x5605ae6bb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdf4168700 1 -- 172.21.15.191:0/2970575316 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5605ae44d340 con 0x5605ae3d3400 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdfa975700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75ac00 0x5605ae6bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605ae3d3800 0x5605ae6bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.839+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae3d3c00 con 0x5605ae3d3800 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfb977700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605ae3d3800 0x5605ae6bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfa975700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75ac00 0x5605ae6bb600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x5605ae77d770 tx=0x5605ae8100c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfb977700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605ae3d3800 0x5605ae6bbb80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 20 realm 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b000 con 0x5605ae3d3800 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b400 con 0x5605ae3d3800 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.840+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b800 con 0x5605ae3d3800 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605ae75bc00 0x5605ae6bc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b000 con 0x5605ae75bc00 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfb176700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605ae75bc00 0x5605ae6bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfb176700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605ae75bc00 0x5605ae6bc100 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 20 searching for the correct realm 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.841+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b400 con 0x5605ae75bc00 2024-04-27T04:34:46.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfb176700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605ae75b400 0x5605ae6bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b000 con 0x5605ae75b400 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfa975700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605ae75b400 0x5605ae6bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfa975700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605ae75b400 0x5605ae6bcc00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfa975700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605ae75b000 0x5605ae6bd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.842+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.843+0000 7fbdfb977700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605ae75b000 0x5605ae6bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.843+0000 7fbdfb977700 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605ae75b000 0x5605ae6bd180 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.843+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.843+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b400 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.844+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b400 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.845+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.846+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.847+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b400 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.848+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b400 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75bc00 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.849+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b400 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae3d3c00 con 0x5605ae3d3800 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830400 con 0x5605ae3d3800 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830800 con 0x5605ae75bc00 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 zone default found 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 4 Realm: () 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:46.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 zonegroup default 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.850+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830c00 con 0x5605ae75b000 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831000 con 0x5605ae75bc00 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831400 con 0x5605ae75bc00 2024-04-27T04:34:46.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 started sync module instance, tier type = 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.851+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831800 con 0x5605ae75b400 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.852+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.852+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94582401080064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830800 con 0x5605ae75b400 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.852+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831000 con 0x5605ae3d3800 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.853+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94582401080064] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.853+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.853+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94582401081216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831400 con 0x5605ae3d3800 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.853+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830400 con 0x5605ae75bc00 2024-04-27T04:34:46.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.854+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94582401081216] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.855+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.855+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94582401082368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830c00 con 0x5605ae75bc00 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.855+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae3d3c00 con 0x5605ae75b400 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.856+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94582401082368] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.856+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.856+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94582401083520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae75b800 con 0x5605ae75b400 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.856+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae3d3800 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.857+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.857+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94582401084672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831800 con 0x5605ae3d3800 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.857+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830800 con 0x5605ae75b000 2024-04-27T04:34:46.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.857+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94582401083520] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.858+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94582401084672] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.858+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.858+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94582401085824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831c00 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.858+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831000 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.859+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94582401085824] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.859+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.859+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94582401086976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831400 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.859+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830000 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.860+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94582401086976] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.860+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.860+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94582401088128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831800 con 0x5605ae75b000 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.860+0000 7fbdfd1cca80 20 add_watcher() i=0 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=1 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=2 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=3 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=4 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=5 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=6 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94582401088128] v31'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 20 add_watcher() i=7 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.861+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830800 con 0x5605ae3d3800 2024-04-27T04:34:46.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.862+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.862+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94582402678784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831c00 con 0x5605ae3d3800 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94582402678784] v31'110 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbde2144700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfd1cca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfd1cca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfd1cca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfd1cca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ae4743c0 con 0x5605ae831400 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfa975700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfa975700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47092/0 (socket says 172.21.15.191:47092) 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.863+0000 7fbdfa975700 1 -- 172.21.15.191:0/3042367594 learned_addr learned my addr 172.21.15.191:0/3042367594 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.864+0000 7fbdfa975700 1 -- 172.21.15.191:0/3042367594 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3fab60 con 0x5605ae831400 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.864+0000 7fbdfa975700 1 --2- 172.21.15.191:0/3042367594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5605ae7bf380 tx=0x5605ae723f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a75246c9780ab4d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.864+0000 7fbde1943700 1 -- 172.21.15.191:0/3042367594 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae848000 con 0x5605ae831400 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.864+0000 7fbde1943700 1 -- 172.21.15.191:0/3042367594 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae84a000 con 0x5605ae831400 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.864+0000 7fbde1943700 1 -- 172.21.15.191:0/3042367594 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae848160 con 0x5605ae831400 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3042367594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 msgr2=0x5605ae6bc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3042367594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5605ae7bf380 tx=0x5605ae723f80 comp rx=0 tx=0).stop 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3042367594 shutdown_connections 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3042367594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3042367594 >> 172.21.15.191:0/3042367594 conn(0x5605ae831000 msgr2=0x5605ae4c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3042367594 shutdown_connections 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3042367594 wait complete. 2024-04-27T04:34:46.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 Processor -- start 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- start start 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfd1cca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ae474500 con 0x5605ae831400 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfa975700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfa975700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47106/0 (socket says 172.21.15.191:47106) 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.865+0000 7fbdfa975700 1 -- 172.21.15.191:0/3936943276 learned_addr learned my addr 172.21.15.191:0/3936943276 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbdfa975700 1 -- 172.21.15.191:0/3936943276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ae3fad00 con 0x5605ae831400 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbdfa975700 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5605ae83c780 tx=0x5605ae8104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbde0140700 1 -- 172.21.15.191:0/3936943276 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae8482c0 con 0x5605ae831400 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbde0140700 1 -- 172.21.15.191:0/3936943276 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605ae84a180 con 0x5605ae831400 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605ae3fa340 con 0x5605ae831400 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbde0140700 1 -- 172.21.15.191:0/3936943276 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5605ae848000 con 0x5605ae831400 2024-04-27T04:34:46.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605ae3fab60 con 0x5605ae831400 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.866+0000 7fbde0140700 1 -- 172.21.15.191:0/3936943276 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5605af175b80 con 0x5605ae831400 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.867+0000 7fbde0140700 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae8c6000 0x5605ae6bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.867+0000 7fbde0140700 1 -- 172.21.15.191:0/3936943276 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5605ae44d500 con 0x5605ae831400 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.867+0000 7fbdfd1cca80 5 note: GC not initialized 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.867+0000 7fbdfb977700 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae8c6000 0x5605ae6bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.867+0000 7fbdfb977700 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae8c6000 0x5605ae6bd700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5605ae7bfdd0 tx=0x5605ae810360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbddd93b700 20 reqs_thread_entry: start 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbddb937700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbddb115700 20 reqs_thread_entry: start 2024-04-27T04:34:46.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831800 con 0x5605ae3d3800 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.868+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830800 con 0x5605ae75b400 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:46.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6400 con 0x5605ae75b000 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.869+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6800 con 0x5605ae75b400 2024-04-27T04:34:46.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.870+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 foo.buckets [call] v31'17 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.870+0000 7fbdfd1cca80 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.870+0000 7fbdfd1cca80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.870+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6c00 con 0x5605ae75b400 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.871+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.871+0000 7fbdfd1cca80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.871+0000 7fbdfd1cca80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.871+0000 7fbdfd1cca80 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.871+0000 7fbdfd1cca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.871+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c7400 con 0x5605ae75b400 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v31'18 uv18 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfd1cca80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfd1cca80 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:46.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfd1cca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94582402681088 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c7800 con 0x5605ae75b000 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== watch-notify(notify (1) cookie 94582401086976 notify 133143986186 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x5605ae7c1d40 con 0x5605ae75b000 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(68 notify.6 [notify cookie 94582402681088 out=8b] v0'0 uv105 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdf596b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986186 cookie 94582401086976 notifier 4508 bl.length()=338 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdf596b700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdf596b700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.872+0000 7fbdf596b700 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae830800 con 0x5605ae75b000 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.873+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== watch-notify(notify_complete (2) cookie 94582402681088 notify 133143986186 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5605ae88fd40 con 0x5605ae75b000 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.873+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.873+0000 7fbde2144700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.873+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94582402678784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c7c00 con 0x5605ae3d3800 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.874+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94582402678784] v31'111 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.874+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94582401080064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6800 con 0x5605ae75b400 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.875+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94582401080064] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.875+0000 7fbdfd1cca80 20 remove_watcher() i=0 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.875+0000 7fbdfd1cca80 2 removed watcher, disabling cache 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.875+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94582401081216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6c00 con 0x5605ae3d3800 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.876+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94582401081216] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.877+0000 7fbdfd1cca80 20 remove_watcher() i=1 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.877+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94582401082368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6400 con 0x5605ae75bc00 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.877+0000 7fbdfb176700 1 -- 172.21.15.191:0/2970575316 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94582401082368] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b200 con 0x5605ae75bc00 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.877+0000 7fbdfd1cca80 20 remove_watcher() i=2 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.877+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94582401083520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c7800 con 0x5605ae75b400 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.878+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94582401083520] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81b440 con 0x5605ae75b400 2024-04-27T04:34:46.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.878+0000 7fbdfd1cca80 20 remove_watcher() i=3 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.878+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94582401084672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c7c00 con 0x5605ae3d3800 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.879+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94582401084672] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae3d3800 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.879+0000 7fbdfd1cca80 20 remove_watcher() i=4 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.879+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94582401085824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae8c6c00 con 0x5605ae75b000 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.880+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94582401085824] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.880+0000 7fbdfd1cca80 20 remove_watcher() i=5 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.880+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94582401086976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831c00 con 0x5605ae75b000 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.881+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94582401086976] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.881+0000 7fbdfd1cca80 20 remove_watcher() i=6 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.881+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94582401088128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5605ae831800 con 0x5605ae75b000 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfb977700 1 -- 172.21.15.191:0/2970575316 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94582401088128] v31'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5605ae81ab40 con 0x5605ae75b000 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 20 remove_watcher() i=7 2024-04-27T04:34:46.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605ae75b000 msgr2=0x5605ae6bd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5605ae75b000 0x5605ae6bd180 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605ae3d3800 msgr2=0x5605ae6bbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5605ae3d3800 0x5605ae6bbb80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605ae75b400 msgr2=0x5605ae6bcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5605ae75b400 0x5605ae6bcc00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605ae75bc00 msgr2=0x5605ae6bc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5605ae75bc00 0x5605ae6bc100 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75ac00 msgr2=0x5605ae6bb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75ac00 0x5605ae6bb600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x5605ae77d770 tx=0x5605ae8100c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 msgr2=0x5605ae6bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/2970575316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d3400 0x5605ae6bab00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b59b0 tx=0x5605ae7226c0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 reap_dead start 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfa975700 1 -- 172.21.15.191:0/2970575316 reap_dead start 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 shutdown_connections 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 >> 172.21.15.191:0/2970575316 conn(0x5605ae3d2c00 msgr2=0x5605ae4bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 shutdown_connections 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.882+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/2970575316 wait complete. 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae8c6000 msgr2=0x5605ae6bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae8c6000 0x5605ae6bd700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5605ae7bfdd0 tx=0x5605ae810360 comp rx=0 tx=0).stop 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 msgr2=0x5605ae6bc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5605ae83c780 tx=0x5605ae8104e0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 shutdown_connections 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae8c6000 0x5605ae6bd700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3936943276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae831400 0x5605ae6bc680 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 >> 172.21.15.191:0/3936943276 conn(0x5605ae831000 msgr2=0x5605ae4c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 shutdown_connections 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3936943276 wait complete. 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75a000 msgr2=0x5605ae6bb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75a000 0x5605ae6bb080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5605af25eed0 tx=0x5605ae722c60 comp rx=0 tx=0).stop 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 msgr2=0x5605ae6ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x5605ae6b4a50 tx=0x5605ae4c3920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 shutdown_connections 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5605ae75a000 0x5605ae6bb080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 --2- 172.21.15.191:0/3123473124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605ae3d2800 0x5605ae6ba580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 >> 172.21.15.191:0/3123473124 conn(0x5605ae3d2400 msgr2=0x5605ae4bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 shutdown_connections 2024-04-27T04:34:46.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.883+0000 7fbdfd1cca80 1 -- 172.21.15.191:0/3123473124 wait complete. 2024-04-27T04:34:46.910 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T04:34:46.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T04:34:46.910 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-27T04:34:46.963 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:46.964 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:46.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb561eeba80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb561eeba80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f258407500 con 0x55f2580dcc00 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb561eeba80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f25817ea00 con 0x55f2580dc800 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb55f694700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55f2580dcc00 0x55f2581c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60386/0 (socket says 172.21.15.191:60386) 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb55fe95700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb55f694700 1 -- 172.21.15.191:0/138796384 learned_addr learned my addr 172.21.15.191:0/138796384 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb55fe95700 1 -- 172.21.15.191:0/138796384 >> v1:172.21.15.138:6789/0 conn(0x55f2580dcc00 legacy=0x55f2581c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.962+0000 7fb55fe95700 1 -- 172.21.15.191:0/138796384 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258105d40 con 0x55f2580dc800 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb55fe95700 1 --2- 172.21.15.191:0/138796384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55f2583bf9e0 tx=0x55f2581cd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b7cdf36b883b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb55ee93700 1 -- 172.21.15.191:0/138796384 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2581954a0 con 0x55f2580dc800 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb55ee93700 1 -- 172.21.15.191:0/138796384 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f258407800 con 0x55f2580dc800 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb55ee93700 1 -- 172.21.15.191:0/138796384 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f258195600 con 0x55f2580dc800 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 -- 172.21.15.191:0/138796384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 msgr2=0x55f2583c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 --2- 172.21.15.191:0/138796384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55f2583bf9e0 tx=0x55f2581cd920 comp rx=0 tx=0).stop 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 -- 172.21.15.191:0/138796384 shutdown_connections 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 --2- 172.21.15.191:0/138796384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 -- 172.21.15.191:0/138796384 >> 172.21.15.191:0/138796384 conn(0x55f2580dc400 msgr2=0x55f2581c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 -- 172.21.15.191:0/138796384 shutdown_connections 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.963+0000 7fb561eeba80 1 -- 172.21.15.191:0/138796384 wait complete. 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb561eeba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb561eeba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25817e500 con 0x55f2580dc800 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb55fe95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb55fe95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47116/0 (socket says 172.21.15.191:47116) 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb55fe95700 1 -- 172.21.15.191:0/1875507091 learned_addr learned my addr 172.21.15.191:0/1875507091 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.964+0000 7fb55fe95700 1 -- 172.21.15.191:0/1875507091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258105ba0 con 0x55f2580dc800 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb55fe95700 1 --2- 172.21.15.191:0/1875507091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55f258ea7d40 tx=0x55f2581cd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ff92a1c1eb0d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb55e692700 1 -- 172.21.15.191:0/1875507091 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f258195760 con 0x55f2580dc800 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb55e692700 1 -- 172.21.15.191:0/1875507091 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f258407b00 con 0x55f2580dc800 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb55e692700 1 -- 172.21.15.191:0/1875507091 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2581951e0 con 0x55f2580dc800 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 -- 172.21.15.191:0/1875507091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 msgr2=0x55f2583c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1875507091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55f258ea7d40 tx=0x55f2581cd800 comp rx=0 tx=0).stop 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 -- 172.21.15.191:0/1875507091 shutdown_connections 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1875507091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 -- 172.21.15.191:0/1875507091 >> 172.21.15.191:0/1875507091 conn(0x55f2580dc400 msgr2=0x55f2581c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 -- 172.21.15.191:0/1875507091 shutdown_connections 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 -- 172.21.15.191:0/1875507091 wait complete. 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.965+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb561eeba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb561eeba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25817e3c0 con 0x55f2580dc800 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55fe95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55fe95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47128/0 (socket says 172.21.15.191:47128) 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55fe95700 1 -- 172.21.15.191:0/907098040 learned_addr learned my addr 172.21.15.191:0/907098040 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55fe95700 1 -- 172.21.15.191:0/907098040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258105a00 con 0x55f2580dc800 2024-04-27T04:34:46.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55fe95700 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55f258ea7620 tx=0x55f2581cd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55ce8f700 1 -- 172.21.15.191:0/907098040 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2581954a0 con 0x55f2580dc800 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55ce8f700 1 -- 172.21.15.191:0/907098040 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f258407c80 con 0x55f2580dc800 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f258105860 con 0x55f2580dc800 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb55ce8f700 1 -- 172.21.15.191:0/907098040 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f258195600 con 0x55f2580dc800 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.966+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2581056c0 con 0x55f2580dc800 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb55ce8f700 1 -- 172.21.15.191:0/907098040 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f258e7eb00 con 0x55f2580dc800 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb561eeba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb561eeba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f258118640 con 0x55f2580dd400 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb55ce8f700 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846e000 0x55f2583c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb560696700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb560696700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47130/0 (socket says 172.21.15.191:47130) 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb560696700 1 -- 172.21.15.191:0/3513691163 learned_addr learned my addr 172.21.15.191:0/3513691163 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb55ce8f700 1 -- 172.21.15.191:0/907098040 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f258157180 con 0x55f2580dc800 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb55fe95700 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846e000 0x55f2583c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.967+0000 7fb560696700 1 -- 172.21.15.191:0/3513691163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258104000 con 0x55f2580dd400 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb560696700 1 --2- 172.21.15.191:0/3513691163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55f258ea7f80 tx=0x55f258434b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eca3dd98451310 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb55fe95700 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846e000 0x55f2583c5080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55f258f72ed0 tx=0x55f258434c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb55a68a700 1 -- 172.21.15.191:0/3513691163 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2581954a0 con 0x55f2580dd400 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb55a68a700 1 -- 172.21.15.191:0/3513691163 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f25817dc80 con 0x55f2580dd400 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb55a68a700 1 -- 172.21.15.191:0/3513691163 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f258195600 con 0x55f2580dd400 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 -- 172.21.15.191:0/3513691163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 msgr2=0x55f2583c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 --2- 172.21.15.191:0/3513691163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55f258ea7f80 tx=0x55f258434b40 comp rx=0 tx=0).stop 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 -- 172.21.15.191:0/3513691163 shutdown_connections 2024-04-27T04:34:46.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 --2- 172.21.15.191:0/3513691163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 -- 172.21.15.191:0/3513691163 >> 172.21.15.191:0/3513691163 conn(0x55f2580dcc00 msgr2=0x55f2581c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 -- 172.21.15.191:0/3513691163 shutdown_connections 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.968+0000 7fb561eeba80 1 -- 172.21.15.191:0/3513691163 wait complete. 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb561eeba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb561eeba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2581188c0 con 0x55f2580dd400 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb560696700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb560696700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47144/0 (socket says 172.21.15.191:47144) 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 learned_addr learned my addr 172.21.15.191:0/2197345560 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258104ea0 con 0x55f2580dd400 2024-04-27T04:34:46.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb560696700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55f258f720f0 tx=0x55f2584345a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb558e87700 1 -- 172.21.15.191:0/2197345560 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f258195760 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb558e87700 1 -- 172.21.15.191:0/2197345560 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f25817db00 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.969+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f258104820 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.970+0000 7fb558e87700 1 -- 172.21.15.191:0/2197345560 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2581954a0 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.970+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2581049c0 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.970+0000 7fb558e87700 1 -- 172.21.15.191:0/2197345560 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f258e7edc0 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.970+0000 7fb558e87700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846ec00 0x55f2583c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.970+0000 7fb558e87700 1 -- 172.21.15.191:0/2197345560 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f258157340 con 0x55f2580dd400 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.970+0000 7fb55f694700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846ec00 0x55f2583c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb55f694700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846ec00 0x55f2583c5600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55f258479770 tx=0x55f2585100c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:46.981 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f2580dd800 0x55f2583c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:46.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2580ddc00 con 0x55f2580dd800 2024-04-27T04:34:46.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb560696700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f2580dd800 0x55f2583c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:46.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb560696700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f2580dd800 0x55f2583c5b80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.971+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 realm 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f000 con 0x55f2580dd800 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f400 con 0x55f2580dd800 2024-04-27T04:34:47.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f800 con 0x55f2580dd800 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f25846fc00 0x55f2583c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f000 con 0x55f25846fc00 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.972+0000 7fb55fe95700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f25846fc00 0x55f2583c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb55fe95700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f25846fc00 0x55f2583c6100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb561eeba80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb561eeba80 20 searching for the correct realm 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f400 con 0x55f25846fc00 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb55fe95700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f25846f400 0x55f2583c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.973+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f000 con 0x55f25846f400 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.974+0000 7fb55f694700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f25846f400 0x55f2583c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.974+0000 7fb55f694700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f25846f400 0x55f2583c6c00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.974+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.974+0000 7fb55f694700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f25846f000 0x55f2583c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.974+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.974+0000 7fb560696700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f25846f000 0x55f2583c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f25846f000 0x55f2583c7180 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.007 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.975+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f400 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.008 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.976+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f400 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.977+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.978+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.010 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f400 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.979+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f400 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.980+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846fc00 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f400 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb561eeba80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb561eeba80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb561eeba80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb561eeba80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.981+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2580ddc00 con 0x55f2580dd800 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e400 con 0x55f2580dd800 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e800 con 0x55f25846fc00 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 zone default found 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 4 Realm: () 2024-04-27T04:34:47.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 zonegroup default 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852ec00 con 0x55f25846f000 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.982+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f000 con 0x55f25846fc00 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f400 con 0x55f25846fc00 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 20 started sync module instance, tier type = 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.983+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f800 con 0x55f25846f400 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.984+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.984+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94499350756096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e800 con 0x55f25846f400 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.984+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f000 con 0x55f2580dd800 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.985+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94499350756096] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.985+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.985+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94499350757248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f400 con 0x55f2580dd800 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.985+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e400 con 0x55f25846fc00 2024-04-27T04:34:47.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.986+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94499350757248] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.987+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.987+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94499350758400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852ec00 con 0x55f25846fc00 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.987+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2580ddc00 con 0x55f25846f400 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.988+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.988+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94499350758400] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.988+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94499350759552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25846f800 con 0x55f25846f400 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.988+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f2580dd800 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.989+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94499350759552] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.989+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.989+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94499350760704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f800 con 0x55f2580dd800 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.990+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e800 con 0x55f25846f000 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.990+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.990+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94499350761856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852fc00 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.990+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f000 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.990+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94499350760704] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.991+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94499350761856] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.991+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.991+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94499350763008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f400 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.991+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e000 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94499350763008] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94499350764160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f800 con 0x55f25846f000 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=0 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=1 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=2 2024-04-27T04:34:47.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=3 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=4 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=5 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.992+0000 7fb561eeba80 20 add_watcher() i=6 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.993+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94499350764160] v31'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.993+0000 7fb561eeba80 20 add_watcher() i=7 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.993+0000 7fb561eeba80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.993+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e800 con 0x55f2580dd800 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.994+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.994+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94499352322048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852fc00 con 0x55f2580dd800 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.994+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94499352322048] v31'112 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb546e63700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb561eeba80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb561eeba80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb561eeba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb561eeba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25817e3c0 con 0x55f25852f400 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb55f694700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb55f694700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47146/0 (socket says 172.21.15.191:47146) 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.995+0000 7fb55f694700 1 -- 172.21.15.191:0/1129505902 learned_addr learned my addr 172.21.15.191:0/1129505902 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb55f694700 1 -- 172.21.15.191:0/1129505902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258104b60 con 0x55f25852f400 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb55f694700 1 --2- 172.21.15.191:0/1129505902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55f2584cf4d0 tx=0x55f258435f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f5f4088d2e4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb546662700 1 -- 172.21.15.191:0/1129505902 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f25854c000 con 0x55f25852f400 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb546662700 1 -- 172.21.15.191:0/1129505902 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f258550000 con 0x55f25852f400 2024-04-27T04:34:47.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb546662700 1 -- 172.21.15.191:0/1129505902 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f25854c160 con 0x55f25852f400 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb561eeba80 1 -- 172.21.15.191:0/1129505902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 msgr2=0x55f2583c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1129505902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55f2584cf4d0 tx=0x55f258435f80 comp rx=0 tx=0).stop 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb561eeba80 1 -- 172.21.15.191:0/1129505902 shutdown_connections 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1129505902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.996+0000 7fb561eeba80 1 -- 172.21.15.191:0/1129505902 >> 172.21.15.191:0/1129505902 conn(0x55f25852f000 msgr2=0x55f2581ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb561eeba80 1 -- 172.21.15.191:0/1129505902 shutdown_connections 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb561eeba80 1 -- 172.21.15.191:0/1129505902 wait complete. 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb561eeba80 1 Processor -- start 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb561eeba80 1 -- start start 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb561eeba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb561eeba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25817e500 con 0x55f25852f400 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb55f694700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb55f694700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47154/0 (socket says 172.21.15.191:47154) 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb55f694700 1 -- 172.21.15.191:0/1053750863 learned_addr learned my addr 172.21.15.191:0/1053750863 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.997+0000 7fb55f694700 1 -- 172.21.15.191:0/1053750863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f258104d00 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb55f694700 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55f25854e210 tx=0x55f2585104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb544e5f700 1 -- 172.21.15.191:0/1053750863 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f25854c2c0 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb544e5f700 1 -- 172.21.15.191:0/1053750863 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f258550180 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f258104340 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb544e5f700 1 -- 172.21.15.191:0/1053750863 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f25854c000 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f258104b60 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb544e5f700 1 -- 172.21.15.191:0/1053750863 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f258e7fb80 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.998+0000 7fb544e5f700 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f2585d0000 0x55f2583c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.999+0000 7fb544e5f700 1 -- 172.21.15.191:0/1053750863 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f258157500 con 0x55f25852f400 2024-04-27T04:34:47.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.999+0000 7fb560696700 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f2585d0000 0x55f2583c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.999+0000 7fb561eeba80 5 note: GC not initialized 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:46.999+0000 7fb560696700 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f2585d0000 0x55f2583c7700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55f2584cfdd0 tx=0x55f258510360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb54265a700 20 reqs_thread_entry: start 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb540656700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb53fe34700 20 reqs_thread_entry: start 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852f800 con 0x55f25846f000 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e800 con 0x55f25846f000 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:34:47.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.000+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d0400 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v31'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb561eeba80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb561eeba80 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb561eeba80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94499352324352 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d0800 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== watch-notify(notify (1) cookie 94499350761856 notify 133143986187 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55f2584cdd40 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb55a68a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986187 cookie 94499350761856 notifier 4516 bl.length()=522 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb55a68a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb55a68a700 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(65 notify.5 [notify cookie 94499352324352 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb55a68a700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f258640000 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== watch-notify(notify_complete (2) cookie 94499352324352 notify 133143986187 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f25859bd40 con 0x55f25846f000 2024-04-27T04:34:47.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.002+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d0c00 con 0x55f25846f000 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'3 uv3 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb561eeba80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb561eeba80 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb561eeba80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94499352324352 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d1000 con 0x55f25846fc00 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94499350758400 notify 133143986178 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55f258429ba0 con 0x55f25846fc00 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb55a68a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94499350758400 notifier 4516 bl.length()=177 2024-04-27T04:34:47.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb55a68a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb55a68a700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(68 notify.2 [notify cookie 94499352324352 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.003+0000 7fb55a68a700 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f258640400 con 0x55f25846fc00 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.004+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.004+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 94499352324352 notify 133143986178 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f258429d40 con 0x55f25846fc00 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "fud", 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Fud", 2024-04-27T04:34:47.027 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "", 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 1, 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "fud", 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:47.028 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:47.029 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:47.030 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.004+0000 7fb546e63700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.004+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94499352322048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d1400 con 0x55f2580dd800 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.005+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94499352322048] v31'113 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.006+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94499350756096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d1800 con 0x55f25846f400 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.007+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94499350756096] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.007+0000 7fb561eeba80 20 remove_watcher() i=0 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.007+0000 7fb561eeba80 2 removed watcher, disabling cache 2024-04-27T04:34:47.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.007+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94499350757248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d1c00 con 0x55f2580dd800 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.008+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94499350757248] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.008+0000 7fb561eeba80 20 remove_watcher() i=1 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.008+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94499350758400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d1000 con 0x55f25846fc00 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.009+0000 7fb55fe95700 1 -- 172.21.15.191:0/2197345560 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94499350758400] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523200 con 0x55f25846fc00 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.009+0000 7fb561eeba80 20 remove_watcher() i=2 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.009+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94499350759552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d0400 con 0x55f25846f400 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.010+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94499350759552] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258523440 con 0x55f25846f400 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.010+0000 7fb561eeba80 20 remove_watcher() i=3 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.010+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94499350760704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d0800 con 0x55f2580dd800 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.011+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94499350760704] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f2580dd800 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.011+0000 7fb561eeba80 20 remove_watcher() i=4 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.011+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94499350761856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f2585d0c00 con 0x55f25846f000 2024-04-27T04:34:47.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.012+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94499350761856] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.012+0000 7fb561eeba80 20 remove_watcher() i=5 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.012+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94499350763008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852e400 con 0x55f25846f000 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.012+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94499350763008] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.012+0000 7fb561eeba80 20 remove_watcher() i=6 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.012+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94499350764160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f25852ec00 con 0x55f25846f000 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb560696700 1 -- 172.21.15.191:0/2197345560 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94499350764160] v31'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f258522b40 con 0x55f25846f000 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 20 remove_watcher() i=7 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f25846f000 msgr2=0x55f2583c7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f25846f000 0x55f2583c7180 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f2580dd800 msgr2=0x55f2583c5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f2580dd800 0x55f2583c5b80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f25846f400 msgr2=0x55f2583c6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f25846f400 0x55f2583c6c00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f25846fc00 msgr2=0x55f2583c6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.013+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f25846fc00 0x55f2583c6100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846ec00 msgr2=0x55f2583c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846ec00 0x55f2583c5600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55f258479770 tx=0x55f2585100c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 msgr2=0x55f2583c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/2197345560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dd400 0x55f2583c4b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55f258f720f0 tx=0x55f2584345a0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 reap_dead start 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 shutdown_connections 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 >> 172.21.15.191:0/2197345560 conn(0x55f2580dcc00 msgr2=0x55f2581c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb55f694700 1 -- 172.21.15.191:0/2197345560 reap_dead start 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 shutdown_connections 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/2197345560 wait complete. 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f2585d0000 msgr2=0x55f2583c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f2585d0000 0x55f2583c7700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55f2584cfdd0 tx=0x55f258510360 comp rx=0 tx=0).stop 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 msgr2=0x55f2583c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55f25854e210 tx=0x55f2585104e0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 shutdown_connections 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f2585d0000 0x55f2583c7700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/1053750863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25852f400 0x55f2583c6680 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 >> 172.21.15.191:0/1053750863 conn(0x55f25852f000 msgr2=0x55f2581ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 shutdown_connections 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/1053750863 wait complete. 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846e000 msgr2=0x55f2583c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.014+0000 7fb561eeba80 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846e000 0x55f2583c5080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55f258f72ed0 tx=0x55f258434c60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 msgr2=0x55f2583c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55f258ea7620 tx=0x55f2581cd920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 shutdown_connections 2024-04-27T04:34:47.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f25846e000 0x55f2583c5080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 --2- 172.21.15.191:0/907098040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2580dc800 0x55f2583c4580 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 >> 172.21.15.191:0/907098040 conn(0x55f2580dc400 msgr2=0x55f2581c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 shutdown_connections 2024-04-27T04:34:47.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.015+0000 7fb561eeba80 1 -- 172.21.15.191:0/907098040 wait complete. 2024-04-27T04:34:47.041 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:47.046 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T04:34:47.050 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:47.051 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-04-27T04:34:47.051 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-04-27T04:34:47.051 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-04-27T04:34:47.096 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.096 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.094+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3dac0a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5648a6a66c00 0x5648a6d4e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3dac0a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5648a6d91500 con 0x5648a6a66800 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3dac0a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5648a6b08a00 con 0x5648a6a66c00 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3ba6a700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5648a6a66800 0x5648a6b52800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60402/0 (socket says 172.21.15.191:60402) 2024-04-27T04:34:47.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3b269700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5648a6a66c00 0x5648a6d4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/4023544959 learned_addr learned my addr 172.21.15.191:0/4023544959 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3b269700 1 -- 172.21.15.191:0/4023544959 >> v1:172.21.15.138:6789/0 conn(0x5648a6a66800 legacy=0x5648a6b52800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3b269700 1 -- 172.21.15.191:0/4023544959 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8fd40 con 0x5648a6a66c00 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.095+0000 7fcb3b269700 1 --2- 172.21.15.191:0/4023544959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66c00 0x5648a6d4e580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5648a6d49890 tx=0x5648a6b57920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23238d56599a8433 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3aa68700 1 -- 172.21.15.191:0/4023544959 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f4a0 con 0x5648a6a66c00 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3aa68700 1 -- 172.21.15.191:0/4023544959 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6d91800 con 0x5648a6a66c00 2024-04-27T04:34:47.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3aa68700 1 -- 172.21.15.191:0/4023544959 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f600 con 0x5648a6a66c00 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/4023544959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66c00 msgr2=0x5648a6d4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/4023544959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66c00 0x5648a6d4e580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5648a6d49890 tx=0x5648a6b57920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/4023544959 shutdown_connections 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/4023544959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66c00 0x5648a6d4e580 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/4023544959 >> 172.21.15.191:0/4023544959 conn(0x5648a6a66400 msgr2=0x5648a6b52000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/4023544959 shutdown_connections 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.096+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/4023544959 wait complete. 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3dac0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3dac0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a6b08500 con 0x5648a6a66800 2024-04-27T04:34:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3ba6a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3ba6a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47164/0 (socket says 172.21.15.191:47164) 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/2183739246 learned_addr learned my addr 172.21.15.191:0/2183739246 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.097+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/2183739246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8fba0 con 0x5648a6a66800 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/2183739246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5648a7831800 tx=0x5648a6b57b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519e45ec9589962e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3a267700 1 -- 172.21.15.191:0/2183739246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f1e0 con 0x5648a6a66800 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3a267700 1 -- 172.21.15.191:0/2183739246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6d91b00 con 0x5648a6a66800 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3a267700 1 -- 172.21.15.191:0/2183739246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f4a0 con 0x5648a6a66800 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2183739246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 msgr2=0x5648a6d4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/2183739246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5648a7831800 tx=0x5648a6b57b60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2183739246 shutdown_connections 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/2183739246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2183739246 >> 172.21.15.191:0/2183739246 conn(0x5648a6a66400 msgr2=0x5648a6b53000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2183739246 shutdown_connections 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2183739246 wait complete. 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.098+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3dac0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3dac0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a6b083c0 con 0x5648a6a66800 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3ba6a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3ba6a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47176/0 (socket says 172.21.15.191:47176) 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/541402925 learned_addr learned my addr 172.21.15.191:0/541402925 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/541402925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8fa00 con 0x5648a6a66800 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5648a7831a10 tx=0x5648a6b57a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb38a64700 1 -- 172.21.15.191:0/541402925 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f600 con 0x5648a6a66800 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb38a64700 1 -- 172.21.15.191:0/541402925 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6b07c80 con 0x5648a6a66800 2024-04-27T04:34:47.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5648a6a8f860 con 0x5648a6a66800 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb38a64700 1 -- 172.21.15.191:0/541402925 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f1e0 con 0x5648a6a66800 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.099+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648a6a8f6c0 con 0x5648a6a66800 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb38a64700 1 -- 172.21.15.191:0/541402925 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5648a7808b00 con 0x5648a6a66800 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3dac0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3dac0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a6b08000 con 0x5648a6a67400 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb38a64700 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6000 0x5648a6d4f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb38a64700 1 -- 172.21.15.191:0/541402925 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5648a6ae1180 con 0x5648a6a66800 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3c26b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3c26b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47190/0 (socket says 172.21.15.191:47190) 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3c26b700 1 -- 172.21.15.191:0/2251263971 learned_addr learned my addr 172.21.15.191:0/2251263971 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6000 0x5648a6d4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.100+0000 7fcb3c26b700 1 -- 172.21.15.191:0/2251263971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8e000 con 0x5648a6a67400 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6000 0x5648a6d4f080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5648a78e0f90 tx=0x5648a6db0240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/2251263971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x5648a78319b0 tx=0x5648a6db0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c962bc72bcfc6fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3625f700 1 -- 172.21.15.191:0/2251263971 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f600 con 0x5648a6a67400 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3625f700 1 -- 172.21.15.191:0/2251263971 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6d90300 con 0x5648a6a67400 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3625f700 1 -- 172.21.15.191:0/2251263971 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f1e0 con 0x5648a6a67400 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2251263971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 msgr2=0x5648a6d4eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/2251263971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x5648a78319b0 tx=0x5648a6db0de0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2251263971 shutdown_connections 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/2251263971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2251263971 >> 172.21.15.191:0/2251263971 conn(0x5648a6a66c00 msgr2=0x5648a6b53800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2251263971 shutdown_connections 2024-04-27T04:34:47.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2251263971 wait complete. 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.101+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3dac0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3dac0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a6aa2640 con 0x5648a6a67400 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3c26b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3c26b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47204/0 (socket says 172.21.15.191:47204) 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 learned_addr learned my addr 172.21.15.191:0/3478693744 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8eea0 con 0x5648a6a67400 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5648a78e0930 tx=0x5648a6b57f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb34a5c700 1 -- 172.21.15.191:0/3478693744 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1fa20 con 0x5648a6a67400 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb34a5c700 1 -- 172.21.15.191:0/3478693744 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6b07c80 con 0x5648a6a67400 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5648a6a8e820 con 0x5648a6a67400 2024-04-27T04:34:47.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb34a5c700 1 -- 172.21.15.191:0/3478693744 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f600 con 0x5648a6a67400 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.102+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648a6a8e9c0 con 0x5648a6a67400 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb34a5c700 1 -- 172.21.15.191:0/3478693744 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5648a7808dc0 con 0x5648a6a67400 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb34a5c700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6c00 0x5648a6d4f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb34a5c700 1 -- 172.21.15.191:0/3478693744 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5648a6ae1340 con 0x5648a6a67400 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb3b269700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6c00 0x5648a6d4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5648a6a67800 0x5648a6d4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6a67c00 con 0x5648a6a67800 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.103+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5648a6a67800 0x5648a6d4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3b269700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6c00 0x5648a6d4f600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5648a6d49890 tx=0x5648a6b57800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5648a6a67800 0x5648a6d4fb80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3dac0a80 20 realm 2024-04-27T04:34:47.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.104+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7000 con 0x5648a6a67800 2024-04-27T04:34:47.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7400 con 0x5648a6a67800 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7800 con 0x5648a6a67800 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5648a6df7c00 0x5648a6d50100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.106+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7000 con 0x5648a6df7c00 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5648a6df7c00 0x5648a6d50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5648a6df7c00 0x5648a6d50100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3dac0a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3dac0a80 20 searching for the correct realm 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7400 con 0x5648a6df7c00 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3ba6a700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5648a6df7400 0x5648a6d50c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.107+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7000 con 0x5648a6df7400 2024-04-27T04:34:47.131 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.108+0000 7fcb3b269700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5648a6df7400 0x5648a6d50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.108+0000 7fcb3b269700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5648a6df7400 0x5648a6d50c00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.108+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.108+0000 7fcb3b269700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5648a6df7000 0x5648a6d51180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.108+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.108+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5648a6df7000 0x5648a6d51180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5648a6df7000 0x5648a6d51180 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.132 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.109+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7400 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.110+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.133 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7400 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.111+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.112+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7400 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.113+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.114+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.115+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7400 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.115+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.115+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7c00 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.115+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.115+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7400 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6a67c00 con 0x5648a6a67800 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8400 con 0x5648a6a67800 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8800 con 0x5648a6df7c00 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 zone default found 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 4 Realm: () 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 zonegroup default 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8c00 con 0x5648a6df7000 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9000 con 0x5648a6df7c00 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9400 con 0x5648a6df7c00 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.117+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.118+0000 7fcb3dac0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.118+0000 7fcb3dac0a80 20 started sync module instance, tier type = 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.118+0000 7fcb3dac0a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.118+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9800 con 0x5648a6df7400 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.119+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.119+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94870036568832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8800 con 0x5648a6df7400 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.119+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9000 con 0x5648a6a67800 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.120+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94870036568832] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.120+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.120+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94870036569984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9400 con 0x5648a6a67800 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.120+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8400 con 0x5648a6df7c00 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.121+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94870036569984] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.121+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.121+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94870036571136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8c00 con 0x5648a6df7c00 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.121+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6a67c00 con 0x5648a6df7400 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.122+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.122+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94870036572288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6df7800 con 0x5648a6df7400 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.122+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94870036571136] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.122+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6a67800 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.123+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94870036572288] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.123+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.123+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94870036573440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9800 con 0x5648a6a67800 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.123+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8800 con 0x5648a6df7000 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.124+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.124+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94870036574592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9c00 con 0x5648a6df7000 2024-04-27T04:34:47.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.124+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9000 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.124+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94870036573440] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.125+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94870036574592] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.125+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.125+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94870036575744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9400 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.125+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8000 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94870036575744] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94870036576896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9800 con 0x5648a6df7000 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=0 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=1 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=2 2024-04-27T04:34:47.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=3 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=4 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=5 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.126+0000 7fcb3dac0a80 20 add_watcher() i=6 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.127+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94870036576896] v31'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.127+0000 7fcb3dac0a80 20 add_watcher() i=7 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.127+0000 7fcb3dac0a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.127+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8800 con 0x5648a6a67800 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.127+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.127+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94870038142976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9c00 con 0x5648a6a67800 2024-04-27T04:34:47.153 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.128+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94870038142976] v31'114 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.153 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.128+0000 7fcb22a38700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.153 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.128+0000 7fcb3dac0a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.153 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.128+0000 7fcb3dac0a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.153 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.128+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3dac0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3dac0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a6aa2780 con 0x5648a6eb9400 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3b269700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3b269700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47218/0 (socket says 172.21.15.191:47218) 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3b269700 1 -- 172.21.15.191:0/2117742008 learned_addr learned my addr 172.21.15.191:0/2117742008 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3b269700 1 -- 172.21.15.191:0/2117742008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8eb60 con 0x5648a6eb9400 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb3b269700 1 --2- 172.21.15.191:0/2117742008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5648a6ece060 tx=0x5648a6b57980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec0e9af1b8fdee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb22237700 1 -- 172.21.15.191:0/2117742008 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6b1f760 con 0x5648a6eb9400 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb22237700 1 -- 172.21.15.191:0/2117742008 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6ed0000 con 0x5648a6eb9400 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.129+0000 7fcb22237700 1 -- 172.21.15.191:0/2117742008 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6eda000 con 0x5648a6eb9400 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2117742008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 msgr2=0x5648a6d50680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/2117742008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5648a6ece060 tx=0x5648a6b57980 comp rx=0 tx=0).stop 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2117742008 shutdown_connections 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/2117742008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2117742008 >> 172.21.15.191:0/2117742008 conn(0x5648a6eb9000 msgr2=0x5648a6b54000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2117742008 shutdown_connections 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/2117742008 wait complete. 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.130+0000 7fcb3dac0a80 1 Processor -- start 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3dac0a80 1 -- start start 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3dac0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3dac0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a6b08000 con 0x5648a6eb9400 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3b269700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3b269700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47232/0 (socket says 172.21.15.191:47232) 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3b269700 1 -- 172.21.15.191:0/30023264 learned_addr learned my addr 172.21.15.191:0/30023264 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3b269700 1 -- 172.21.15.191:0/30023264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a6a8ed00 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3b269700 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5648a6ece270 tx=0x5648a6b578c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb20a34700 1 -- 172.21.15.191:0/30023264 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6eda160 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb20a34700 1 -- 172.21.15.191:0/30023264 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648a6ed0180 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb20a34700 1 -- 172.21.15.191:0/30023264 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5648a6eda2c0 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5648a6a8e340 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.131+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648a6a8fd40 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.132+0000 7fcb20a34700 1 -- 172.21.15.191:0/30023264 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5648a7809b80 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.132+0000 7fcb20a34700 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6f52000 0x5648a6d51700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.132+0000 7fcb20a34700 1 -- 172.21.15.191:0/30023264 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5648a6ae1500 con 0x5648a6eb9400 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.132+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6f52000 0x5648a6d51700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.132+0000 7fcb3dac0a80 5 note: GC not initialized 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.133+0000 7fcb3c26b700 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6f52000 0x5648a6d51700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5648a6e59ce0 tx=0x5648a6edb200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.133+0000 7fcb1e22f700 20 reqs_thread_entry: start 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.133+0000 7fcb3dac0a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.133+0000 7fcb3dac0a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.133+0000 7fcb1c22b700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.133+0000 7fcb1ba09700 20 reqs_thread_entry: start 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.134+0000 7fcb3dac0a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.134+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb9800 con 0x5648a6df7400 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.134+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.134+0000 7fcb3dac0a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.135+0000 7fcb22a38700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.135+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94870038142976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6eb8800 con 0x5648a6a67800 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.136+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94870038142976] v31'115 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.136+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94870036568832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f52400 con 0x5648a6df7400 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.137+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94870036568832] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.137+0000 7fcb3dac0a80 20 remove_watcher() i=0 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.137+0000 7fcb3dac0a80 2 removed watcher, disabling cache 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.137+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94870036569984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f52800 con 0x5648a6a67800 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.138+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94870036569984] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.138+0000 7fcb3dac0a80 20 remove_watcher() i=1 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.138+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94870036571136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f52c00 con 0x5648a6df7c00 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.139+0000 7fcb3ba6a700 1 -- 172.21.15.191:0/3478693744 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94870036571136] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1200 con 0x5648a6df7c00 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.139+0000 7fcb3dac0a80 20 remove_watcher() i=2 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.139+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94870036572288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f53000 con 0x5648a6df7400 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.140+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94870036572288] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6eb1440 con 0x5648a6df7400 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.140+0000 7fcb3dac0a80 20 remove_watcher() i=3 2024-04-27T04:34:47.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.140+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94870036573440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f53400 con 0x5648a6a67800 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.141+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94870036573440] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6a67800 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.141+0000 7fcb3dac0a80 20 remove_watcher() i=4 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.141+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94870036574592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f53800 con 0x5648a6df7000 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.142+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94870036574592] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.142+0000 7fcb3dac0a80 20 remove_watcher() i=5 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.142+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94870036575744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f53c00 con 0x5648a6df7000 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.143+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94870036575744] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.143+0000 7fcb3dac0a80 20 remove_watcher() i=6 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.143+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94870036576896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5648a6f52400 con 0x5648a6df7000 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3c26b700 1 -- 172.21.15.191:0/3478693744 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94870036576896] v31'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5648a6ea98c0 con 0x5648a6df7000 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 20 remove_watcher() i=7 2024-04-27T04:34:47.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5648a6df7000 msgr2=0x5648a6d51180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5648a6df7000 0x5648a6d51180 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5648a6a67800 msgr2=0x5648a6d4fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5648a6a67800 0x5648a6d4fb80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5648a6df7400 msgr2=0x5648a6d50c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5648a6df7400 0x5648a6d50c00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5648a6df7c00 msgr2=0x5648a6d50100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5648a6df7c00 0x5648a6d50100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6c00 msgr2=0x5648a6d4f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6c00 0x5648a6d4f600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5648a6d49890 tx=0x5648a6b57800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 msgr2=0x5648a6d4eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/3478693744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a67400 0x5648a6d4eb00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5648a78e0930 tx=0x5648a6b57f20 comp rx=0 tx=0).stop 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 reap_dead start 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3b269700 1 -- 172.21.15.191:0/3478693744 reap_dead start 2024-04-27T04:34:47.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 shutdown_connections 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 >> 172.21.15.191:0/3478693744 conn(0x5648a6a66c00 msgr2=0x5648a6b53800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 shutdown_connections 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/3478693744 wait complete. 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6f52000 msgr2=0x5648a6d51700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6f52000 0x5648a6d51700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5648a6e59ce0 tx=0x5648a6edb200 comp rx=0 tx=0).stop 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 msgr2=0x5648a6d50680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5648a6ece270 tx=0x5648a6b578c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 shutdown_connections 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6f52000 0x5648a6d51700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/30023264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6eb9400 0x5648a6d50680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 >> 172.21.15.191:0/30023264 conn(0x5648a6eb9000 msgr2=0x5648a6b54000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 shutdown_connections 2024-04-27T04:34:47.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.144+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/30023264 wait complete. 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6000 msgr2=0x5648a6d4f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6000 0x5648a6d4f080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5648a78e0f90 tx=0x5648a6db0240 comp rx=0 tx=0).stop 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 msgr2=0x5648a6d4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5648a7831a10 tx=0x5648a6b57a40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 shutdown_connections 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5648a6df6000 0x5648a6d4f080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 --2- 172.21.15.191:0/541402925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648a6a66800 0x5648a6d4e580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 >> 172.21.15.191:0/541402925 conn(0x5648a6a66400 msgr2=0x5648a6b53000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 shutdown_connections 2024-04-27T04:34:47.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.145+0000 7fcb3dac0a80 1 -- 172.21.15.191:0/541402925 wait complete. 2024-04-27T04:34:47.163 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:47.163 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "bucket:myfoo", 2024-04-27T04:34:47.163 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": { 2024-04-27T04:34:47.163 INFO:teuthology.orchestra.run.smithi191.stdout: "tag": "_quw69xWM16FY4RfsU7PBNs8", 2024-04-27T04:34:47.163 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": 2 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "mtime": "2024-04-27T04:34:46.873108Z", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "data": { 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": { 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "myfoo", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "marker": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "tenant": "", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "explicit_placement": { 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "", 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "" 2024-04-27T04:34:47.164 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout: "creation_time": "2024-04-27T04:34:45.731629Z", 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout: "linked": false, 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout: "has_bucket_info": false 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:47.165 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:47.165 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_quw69xWM16FY4RfsU7PBNs8', 'ver': 2}, 'mtime': '2024-04-27T04:34:46.873108Z', 'data': {'bucket': {'name': 'myfoo', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'bucket_id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-04-27T04:34:45.731629Z', 'linked': False, 'has_bucket_info': False}} 2024-04-27T04:34:47.165 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1'] 2024-04-27T04:34:47.165 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1'] 2024-04-27T04:34:47.166 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:47.214 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.215 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.213+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.213+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.213+0000 7fab657cba80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.213+0000 7fab657cba80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5628bce4d500 con 0x5628bcb22c00 2024-04-27T04:34:47.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.213+0000 7fab657cba80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5628bcbc4a00 con 0x5628bcb22800 2024-04-27T04:34:47.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab63775700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab63775700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47234/0 (socket says 172.21.15.191:47234) 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab62f74700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5628bcb22c00 0x5628bcc0e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60404/0 (socket says 172.21.15.191:60404) 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab63775700 1 -- 172.21.15.191:0/74012832 learned_addr learned my addr 172.21.15.191:0/74012832 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab63775700 1 -- 172.21.15.191:0/74012832 >> v1:172.21.15.138:6789/0 conn(0x5628bcb22c00 legacy=0x5628bcc0e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab63775700 1 -- 172.21.15.191:0/74012832 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4bd40 con 0x5628bcb22800 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab62773700 1 -- 172.21.15.191:0/74012832 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743233129 0 0) 0x5628bd8f0000 con 0x5628bcb22c00 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab63775700 1 --2- 172.21.15.191:0/74012832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x5628bce059e0 tx=0x5628bcc13920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaa11e5a454af40 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab62773700 1 -- 172.21.15.191:0/74012832 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb4a0 con 0x5628bcb22800 2024-04-27T04:34:47.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab62773700 1 -- 172.21.15.191:0/74012832 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bce4d800 con 0x5628bcb22800 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.214+0000 7fab62773700 1 -- 172.21.15.191:0/74012832 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb600 con 0x5628bcb22800 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 -- 172.21.15.191:0/74012832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 msgr2=0x5628bce0a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 --2- 172.21.15.191:0/74012832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x5628bce059e0 tx=0x5628bcc13920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 -- 172.21.15.191:0/74012832 shutdown_connections 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 --2- 172.21.15.191:0/74012832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 -- 172.21.15.191:0/74012832 >> 172.21.15.191:0/74012832 conn(0x5628bcb22400 msgr2=0x5628bcc0e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 -- 172.21.15.191:0/74012832 shutdown_connections 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 -- 172.21.15.191:0/74012832 wait complete. 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.215+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab657cba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab657cba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bcbc4500 con 0x5628bcb22800 2024-04-27T04:34:47.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab63775700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab63775700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47250/0 (socket says 172.21.15.191:47250) 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab63775700 1 -- 172.21.15.191:0/507439517 learned_addr learned my addr 172.21.15.191:0/507439517 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab63775700 1 -- 172.21.15.191:0/507439517 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4bba0 con 0x5628bcb22800 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab63775700 1 --2- 172.21.15.191:0/507439517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5628bce05f50 tx=0x5628bcc13800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b206f5a505c91ac server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab61f72700 1 -- 172.21.15.191:0/507439517 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb760 con 0x5628bcb22800 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab61f72700 1 -- 172.21.15.191:0/507439517 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bce4db00 con 0x5628bcb22800 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.216+0000 7fab61f72700 1 -- 172.21.15.191:0/507439517 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb1e0 con 0x5628bcb22800 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- 172.21.15.191:0/507439517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 msgr2=0x5628bce0a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 --2- 172.21.15.191:0/507439517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5628bce05f50 tx=0x5628bcc13800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- 172.21.15.191:0/507439517 shutdown_connections 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 --2- 172.21.15.191:0/507439517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- 172.21.15.191:0/507439517 >> 172.21.15.191:0/507439517 conn(0x5628bcb22400 msgr2=0x5628bcc0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- 172.21.15.191:0/507439517 shutdown_connections 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- 172.21.15.191:0/507439517 wait complete. 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.217+0000 7fab657cba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bcbc43c0 con 0x5628bcb22800 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab63775700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab63775700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47264/0 (socket says 172.21.15.191:47264) 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab63775700 1 -- 172.21.15.191:0/358874927 learned_addr learned my addr 172.21.15.191:0/358874927 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab63775700 1 -- 172.21.15.191:0/358874927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4ba00 con 0x5628bcb22800 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab63775700 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x5628bd8e9f50 tx=0x5628bcc13920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab6076f700 1 -- 172.21.15.191:0/358874927 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb4a0 con 0x5628bcb22800 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab6076f700 1 -- 172.21.15.191:0/358874927 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bce4dc80 con 0x5628bcb22800 2024-04-27T04:34:47.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628bcb4b860 con 0x5628bcb22800 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab6076f700 1 -- 172.21.15.191:0/358874927 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb600 con 0x5628bcb22800 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.218+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628bcb4b6c0 con 0x5628bcb22800 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab6076f700 1 -- 172.21.15.191:0/358874927 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5628bd8c4b00 con 0x5628bcb22800 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab657cba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab657cba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bcb5e640 con 0x5628bcb23400 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab6076f700 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceac000 0x5628bce0b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab6076f700 1 -- 172.21.15.191:0/358874927 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5628bcb9d180 con 0x5628bcb22800 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63f76700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63f76700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47268/0 (socket says 172.21.15.191:47268) 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63f76700 1 -- 172.21.15.191:0/1232321224 learned_addr learned my addr 172.21.15.191:0/1232321224 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63775700 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceac000 0x5628bce0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63f76700 1 -- 172.21.15.191:0/1232321224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4a000 con 0x5628bcb23400 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63775700 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceac000 0x5628bce0b080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5628bd9aeed0 tx=0x5628bce72c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab63f76700 1 --2- 172.21.15.191:0/1232321224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5628bd8e9e00 tx=0x5628bce72b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facb9a20aebd43ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab5df6a700 1 -- 172.21.15.191:0/1232321224 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb4a0 con 0x5628bcb23400 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab5df6a700 1 -- 172.21.15.191:0/1232321224 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bcbc3c80 con 0x5628bcb23400 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.219+0000 7fab5df6a700 1 -- 172.21.15.191:0/1232321224 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb600 con 0x5628bcb23400 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- 172.21.15.191:0/1232321224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 msgr2=0x5628bce0ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 --2- 172.21.15.191:0/1232321224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5628bd8e9e00 tx=0x5628bce72b40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- 172.21.15.191:0/1232321224 shutdown_connections 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 --2- 172.21.15.191:0/1232321224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- 172.21.15.191:0/1232321224 >> 172.21.15.191:0/1232321224 conn(0x5628bcb22c00 msgr2=0x5628bcc0f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- 172.21.15.191:0/1232321224 shutdown_connections 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- 172.21.15.191:0/1232321224 wait complete. 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.220+0000 7fab657cba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bcb5e8c0 con 0x5628bcb23400 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab63f76700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab63f76700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47270/0 (socket says 172.21.15.191:47270) 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 learned_addr learned my addr 172.21.15.191:0/2666061760 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4aea0 con 0x5628bcb23400 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab63f76700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5628bd9ae810 tx=0x5628bcc13ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab5c767700 1 -- 172.21.15.191:0/2666061760 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb760 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab5c767700 1 -- 172.21.15.191:0/2666061760 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bcbc3b00 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628bcb4a820 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab5c767700 1 -- 172.21.15.191:0/2666061760 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcbdb4a0 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.221+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628bcb4a9c0 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab5c767700 1 -- 172.21.15.191:0/2666061760 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5628bd8c4dc0 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab5c767700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceacc00 0x5628bce0b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab5c767700 1 -- 172.21.15.191:0/2666061760 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5628bcb9d340 con 0x5628bcb23400 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab62f74700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceacc00 0x5628bce0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5628bcb23800 0x5628bce0bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcb23c00 con 0x5628bcb23800 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.222+0000 7fab63f76700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5628bcb23800 0x5628bce0bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab62f74700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceacc00 0x5628bce0b600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x5628bce91b00 tx=0x5628bcf5e0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab63f76700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5628bcb23800 0x5628bce0bb80 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 realm 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead000 con 0x5628bcb23800 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead400 con 0x5628bcb23800 2024-04-27T04:34:47.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.223+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead800 con 0x5628bcb23800 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5628bceadc00 0x5628bce0c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead000 con 0x5628bceadc00 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab63775700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5628bceadc00 0x5628bce0c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab63775700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5628bceadc00 0x5628bce0c100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 20 searching for the correct realm 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.224+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead400 con 0x5628bceadc00 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab63775700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5628bcead400 0x5628bce0cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead000 con 0x5628bcead400 2024-04-27T04:34:47.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab62f74700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5628bcead400 0x5628bce0cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab62f74700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5628bcead400 0x5628bce0cc00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab62f74700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5628bcead000 0x5628bce0d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.225+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.226+0000 7fab63f76700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5628bcead000 0x5628bce0d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.226+0000 7fab63f76700 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5628bcead000 0x5628bce0d180 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.226+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.226+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.226+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.226+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead400 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.227+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead400 2024-04-27T04:34:47.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.228+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.229+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead400 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.230+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead400 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.231+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bceadc00 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead400 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab657cba80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab657cba80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab657cba80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab657cba80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.232+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcb23c00 con 0x5628bcb23800 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78400 con 0x5628bcb23800 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78800 con 0x5628bceadc00 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 zone default found 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 4 Realm: () 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 zonegroup default 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78c00 con 0x5628bcead000 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.233+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79000 con 0x5628bceadc00 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79400 con 0x5628bceadc00 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 20 started sync module instance, tier type = 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.234+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79800 con 0x5628bcead400 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.235+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.235+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94732967484160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78800 con 0x5628bcead400 2024-04-27T04:34:47.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.235+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79000 con 0x5628bcb23800 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.236+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94732967484160] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.236+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.236+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94732967485312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79400 con 0x5628bcb23800 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.236+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78400 con 0x5628bceadc00 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.237+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94732967485312] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.237+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.237+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94732967486464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78c00 con 0x5628bceadc00 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.237+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcb23c00 con 0x5628bcead400 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.238+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94732967486464] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.238+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.238+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94732967487616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcead800 con 0x5628bcead400 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.239+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcb23800 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.239+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.239+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94732967488768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79800 con 0x5628bcb23800 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.239+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78800 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.240+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94732967487616] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.240+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.241+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94732967488768] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.241+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94732967489920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79c00 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.241+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79000 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.241+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94732967489920] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.241+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.241+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94732967491072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79400 con 0x5628bcead000 2024-04-27T04:34:47.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78000 con 0x5628bcead000 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94732967491072] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94732967492224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79800 con 0x5628bcead000 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=0 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=1 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=2 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=3 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=4 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=5 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.242+0000 7fab657cba80 20 add_watcher() i=6 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.243+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94732967492224] v31'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.243+0000 7fab657cba80 20 add_watcher() i=7 2024-04-27T04:34:47.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.243+0000 7fab657cba80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.243+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78800 con 0x5628bcb23800 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.244+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.244+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94732969066496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79c00 con 0x5628bcb23800 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.244+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94732969066496] v31'116 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab4a743700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab657cba80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab657cba80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab657cba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab657cba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bcbc43c0 con 0x5628bcf79400 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab62f74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab62f74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47286/0 (socket says 172.21.15.191:47286) 2024-04-27T04:34:47.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab62f74700 1 -- 172.21.15.191:0/3962104770 learned_addr learned my addr 172.21.15.191:0/3962104770 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.245+0000 7fab62f74700 1 -- 172.21.15.191:0/3962104770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4ab60 con 0x5628bcf79400 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab62f74700 1 --2- 172.21.15.191:0/3962104770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5628bce91230 tx=0x5628bce73f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89ef1deb3bad89f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab49f42700 1 -- 172.21.15.191:0/3962104770 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcf96000 con 0x5628bcf79400 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab49f42700 1 -- 172.21.15.191:0/3962104770 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bcfa0000 con 0x5628bcf79400 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab49f42700 1 -- 172.21.15.191:0/3962104770 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcf96160 con 0x5628bcf79400 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab657cba80 1 -- 172.21.15.191:0/3962104770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 msgr2=0x5628bce0c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab657cba80 1 --2- 172.21.15.191:0/3962104770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5628bce91230 tx=0x5628bce73f80 comp rx=0 tx=0).stop 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab657cba80 1 -- 172.21.15.191:0/3962104770 shutdown_connections 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab657cba80 1 --2- 172.21.15.191:0/3962104770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab657cba80 1 -- 172.21.15.191:0/3962104770 >> 172.21.15.191:0/3962104770 conn(0x5628bcf79000 msgr2=0x5628bcc10000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.246+0000 7fab657cba80 1 -- 172.21.15.191:0/3962104770 shutdown_connections 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab657cba80 1 -- 172.21.15.191:0/3962104770 wait complete. 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab657cba80 1 Processor -- start 2024-04-27T04:34:47.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab657cba80 1 -- start start 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab657cba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab657cba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bcbc4500 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab62f74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab62f74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47302/0 (socket says 172.21.15.191:47302) 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab62f74700 1 -- 172.21.15.191:0/561535722 learned_addr learned my addr 172.21.15.191:0/561535722 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.247+0000 7fab62f74700 1 -- 172.21.15.191:0/561535722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bcb4ad00 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab62f74700 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x5628bcf98210 tx=0x5628bcf5e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab4873f700 1 -- 172.21.15.191:0/561535722 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcf962c0 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab4873f700 1 -- 172.21.15.191:0/561535722 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628bcfa0180 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628bcb4a340 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab4873f700 1 -- 172.21.15.191:0/561535722 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628bcf96000 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628bcb4ab60 con 0x5628bcf79400 2024-04-27T04:34:47.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.248+0000 7fab4873f700 1 -- 172.21.15.191:0/561535722 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5628bd8c5b80 con 0x5628bcf79400 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.249+0000 7fab4873f700 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bd01a000 0x5628bce0d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.249+0000 7fab4873f700 1 -- 172.21.15.191:0/561535722 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5628bcb9d500 con 0x5628bcf79400 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.249+0000 7fab63f76700 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bd01a000 0x5628bce0d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.249+0000 7fab657cba80 5 note: GC not initialized 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.249+0000 7fab63f76700 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bd01a000 0x5628bce0d700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5628bcf1de90 tx=0x5628bcf5e360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab45f3a700 20 reqs_thread_entry: start 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab657cba80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab657cba80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab43f36700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab43714700 20 reqs_thread_entry: start 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab657cba80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.250+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79800 con 0x5628bcead000 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78800 con 0x5628bcead400 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:47.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01a400 con 0x5628bcead000 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.251+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01a800 con 0x5628bcead400 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.252+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 foo.buckets [call] v31'19 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.253+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01b000 con 0x5628bcead000 2024-04-27T04:34:47.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(66 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v31'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 10 moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [notify cookie 94732969068800 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01b800 con 0x5628bcead400 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify (1) cookie 94732967487616 notify 133143986189 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5628bcf141a0 con 0x5628bcead400 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(67 notify.3 [notify cookie 94732969068800 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab5df6a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986189 cookie 94732967487616 notifier 4532 bl.length()=725 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab5df6a700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab5df6a700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab5df6a700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab5df6a700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf78800 con 0x5628bcead400 2024-04-27T04:34:47.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(68 notify.3 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== watch-notify(notify_complete (2) cookie 94732969068800 notify 133143986189 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5628bcf14340 con 0x5628bcead400 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_sources= new_sources= 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential sources added=[] removed=[] 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_dests= new_dests= 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential dests added=[] removed=[] 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.254+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01bc00 con 0x5628bcead000 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.255+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(69 fud.buckets [call] v31'1 uv1 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.255+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01a400 con 0x5628bcead400 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v31'20 uv20 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab657cba80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab657cba80 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab657cba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94732969068800 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01b400 con 0x5628bcead000 2024-04-27T04:34:47.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== watch-notify(notify (1) cookie 94732967491072 notify 133143986188 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x5628bcf15d40 con 0x5628bcead000 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab5df6a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986188 cookie 94732967491072 notifier 4532 bl.length()=314 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(71 notify.6 [notify cookie 94732969068800 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab5df6a700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab5df6a700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab5df6a700 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcb23c00 con 0x5628bcead000 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.257+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== watch-notify(notify_complete (2) cookie 94732969068800 notify 133143986188 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5628bcf15a00 con 0x5628bcead000 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.258+0000 7fab4a743700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.258+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94732969066496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01ac00 con 0x5628bcb23800 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.259+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94732969066496] v31'117 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.259+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94732967484160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01b000 con 0x5628bcead400 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.260+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94732967484160] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.260+0000 7fab657cba80 20 remove_watcher() i=0 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.260+0000 7fab657cba80 2 removed watcher, disabling cache 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.260+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94732967485312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01bc00 con 0x5628bcb23800 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.261+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94732967485312] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.261+0000 7fab657cba80 20 remove_watcher() i=1 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.261+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94732967486464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01a800 con 0x5628bceadc00 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.262+0000 7fab63775700 1 -- 172.21.15.191:0/2666061760 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94732967486464] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b200 con 0x5628bceadc00 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.262+0000 7fab657cba80 20 remove_watcher() i=2 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.262+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94732967487616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bd01b800 con 0x5628bcead400 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.263+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94732967487616] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6b440 con 0x5628bcead400 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.263+0000 7fab657cba80 20 remove_watcher() i=3 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.263+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94732967488768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79c00 con 0x5628bcb23800 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.264+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94732967488768] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcb23800 2024-04-27T04:34:47.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.264+0000 7fab657cba80 20 remove_watcher() i=4 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.264+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94732967489920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf79800 con 0x5628bcead000 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.265+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94732967489920] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.265+0000 7fab657cba80 20 remove_watcher() i=5 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.265+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94732967491072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf4a000 con 0x5628bcead000 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.266+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94732967491072] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.266+0000 7fab657cba80 20 remove_watcher() i=6 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.266+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94732967492224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628bcf4a400 con 0x5628bcead000 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab63f76700 1 -- 172.21.15.191:0/2666061760 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94732967492224] v31'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628bcf6ab40 con 0x5628bcead000 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 20 remove_watcher() i=7 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5628bcead000 msgr2=0x5628bce0d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5628bcead000 0x5628bce0d180 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5628bcb23800 msgr2=0x5628bce0bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5628bcb23800 0x5628bce0bb80 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5628bcead400 msgr2=0x5628bce0cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5628bcead400 0x5628bce0cc00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5628bceadc00 msgr2=0x5628bce0c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5628bceadc00 0x5628bce0c100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceacc00 msgr2=0x5628bce0b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceacc00 0x5628bce0b600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x5628bce91b00 tx=0x5628bcf5e0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 msgr2=0x5628bce0ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 --2- 172.21.15.191:0/2666061760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb23400 0x5628bce0ab00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5628bd9ae810 tx=0x5628bcc13ce0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 reap_dead start 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab62f74700 1 -- 172.21.15.191:0/2666061760 reap_dead start 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 shutdown_connections 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 >> 172.21.15.191:0/2666061760 conn(0x5628bcb22c00 msgr2=0x5628bcc0f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 shutdown_connections 2024-04-27T04:34:47.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.267+0000 7fab657cba80 1 -- 172.21.15.191:0/2666061760 wait complete. 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bd01a000 msgr2=0x5628bce0d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bd01a000 0x5628bce0d700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5628bcf1de90 tx=0x5628bcf5e360 comp rx=0 tx=0).stop 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 msgr2=0x5628bce0c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x5628bcf98210 tx=0x5628bcf5e4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 shutdown_connections 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bd01a000 0x5628bce0d700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/561535722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcf79400 0x5628bce0c680 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 >> 172.21.15.191:0/561535722 conn(0x5628bcf79000 msgr2=0x5628bcc10000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 shutdown_connections 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/561535722 wait complete. 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceac000 msgr2=0x5628bce0b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceac000 0x5628bce0b080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5628bd9aeed0 tx=0x5628bce72c60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 msgr2=0x5628bce0a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x5628bd8e9f50 tx=0x5628bcc13920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 shutdown_connections 2024-04-27T04:34:47.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5628bceac000 0x5628bce0b080 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 --2- 172.21.15.191:0/358874927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bcb22800 0x5628bce0a580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 >> 172.21.15.191:0/358874927 conn(0x5628bcb22400 msgr2=0x5628bcc0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 shutdown_connections 2024-04-27T04:34:47.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.268+0000 7fab657cba80 1 -- 172.21.15.191:0/358874927 wait complete. 2024-04-27T04:34:47.292 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T04:34:47.292 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T04:34:47.293 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T04:34:47.340 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.340 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.336+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.336+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.336+0000 7fd66a39aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x558be64d6800 0x558be67be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.336+0000 7fd66a39aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558be6801500 con 0x558be64d6c00 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.336+0000 7fd66a39aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x558be6578a00 con 0x558be64d6800 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd668344700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x558be64d6800 0x558be67be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd667b43700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x558be64d6c00 0x558be65c2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60408/0 (socket says 172.21.15.191:60408) 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd668344700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x558be64d6800 0x558be67be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47318/0 (socket says 172.21.15.191:47318) 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd667b43700 1 -- 172.21.15.191:0/1085365046 learned_addr learned my addr 172.21.15.191:0/1085365046 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd668344700 1 -- 172.21.15.191:0/1085365046 >> v1:172.21.15.138:6789/0 conn(0x558be64d6c00 legacy=0x558be65c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd668344700 1 -- 172.21.15.191:0/1085365046 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64ffd40 con 0x558be64d6800 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd668344700 1 --2- 172.21.15.191:0/1085365046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x558be67b99e0 tx=0x558be65c7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d03599c59fafae server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd667342700 1 -- 172.21.15.191:0/1085365046 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f4a0 con 0x558be64d6800 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd667342700 1 -- 172.21.15.191:0/1085365046 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6801800 con 0x558be64d6800 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.337+0000 7fd667342700 1 -- 172.21.15.191:0/1085365046 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f600 con 0x558be64d6800 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 -- 172.21.15.191:0/1085365046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 msgr2=0x558be67be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/1085365046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x558be67b99e0 tx=0x558be65c7920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 -- 172.21.15.191:0/1085365046 shutdown_connections 2024-04-27T04:34:47.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/1085365046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 -- 172.21.15.191:0/1085365046 >> 172.21.15.191:0/1085365046 conn(0x558be64d6400 msgr2=0x558be65c2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 -- 172.21.15.191:0/1085365046 shutdown_connections 2024-04-27T04:34:47.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 -- 172.21.15.191:0/1085365046 wait complete. 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.338+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd66a39aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd66a39aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be6578500 con 0x558be64d6800 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47328/0 (socket says 172.21.15.191:47328) 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd668344700 1 -- 172.21.15.191:0/4183249246 learned_addr learned my addr 172.21.15.191:0/4183249246 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd668344700 1 -- 172.21.15.191:0/4183249246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64ffba0 con 0x558be64d6800 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd668344700 1 --2- 172.21.15.191:0/4183249246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x558be72ae0c0 tx=0x558be65c7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6460782be162ae47 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd666b41700 1 -- 172.21.15.191:0/4183249246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f760 con 0x558be64d6800 2024-04-27T04:34:47.362 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd666b41700 1 -- 172.21.15.191:0/4183249246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6801b00 con 0x558be64d6800 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.339+0000 7fd666b41700 1 -- 172.21.15.191:0/4183249246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f4a0 con 0x558be64d6800 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- 172.21.15.191:0/4183249246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 msgr2=0x558be67be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/4183249246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x558be72ae0c0 tx=0x558be65c7800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- 172.21.15.191:0/4183249246 shutdown_connections 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/4183249246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- 172.21.15.191:0/4183249246 >> 172.21.15.191:0/4183249246 conn(0x558be64d6400 msgr2=0x558be65c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- 172.21.15.191:0/4183249246 shutdown_connections 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- 172.21.15.191:0/4183249246 wait complete. 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.340+0000 7fd66a39aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be65783c0 con 0x558be64d6800 2024-04-27T04:34:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47330/0 (socket says 172.21.15.191:47330) 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd668344700 1 -- 172.21.15.191:0/121810765 learned_addr learned my addr 172.21.15.191:0/121810765 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd668344700 1 -- 172.21.15.191:0/121810765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64ffa00 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd668344700 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x558be72af980 tx=0x558be65c7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd66533e700 1 -- 172.21.15.191:0/121810765 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f600 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd66533e700 1 -- 172.21.15.191:0/121810765 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6801c80 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd66533e700 1 -- 172.21.15.191:0/121810765 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f8c0 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558be64ff860 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.341+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558be64ff6c0 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66533e700 1 -- 172.21.15.191:0/121810765 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558be7278b00 con 0x558be64d6800 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.364 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66533e700 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be64d7400 0x558be67bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66a39aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66533e700 1 -- 172.21.15.191:0/121810765 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x558be6551180 con 0x558be64d6800 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd66a39aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be6512640 con 0x558be64d7800 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd668b45700 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be64d7400 0x558be67bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47338/0 (socket says 172.21.15.191:47338) 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd668344700 1 -- 172.21.15.191:0/3390969363 learned_addr learned my addr 172.21.15.191:0/3390969363 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd668344700 1 -- 172.21.15.191:0/3390969363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64fe000 con 0x558be64d7800 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.342+0000 7fd668b45700 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be64d7400 0x558be67bf080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x558be72afa10 tx=0x558be6824b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd668344700 1 --2- 172.21.15.191:0/3390969363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x558be7362240 tx=0x558be6824c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e915518948e66dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd662b39700 1 -- 172.21.15.191:0/3390969363 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f600 con 0x558be64d7800 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd662b39700 1 -- 172.21.15.191:0/3390969363 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6801c80 con 0x558be64d7800 2024-04-27T04:34:47.365 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd662b39700 1 -- 172.21.15.191:0/3390969363 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f8c0 con 0x558be64d7800 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3390969363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 msgr2=0x558be67beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3390969363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x558be7362240 tx=0x558be6824c60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3390969363 shutdown_connections 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3390969363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3390969363 >> 172.21.15.191:0/3390969363 conn(0x558be64d6c00 msgr2=0x558be65c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3390969363 shutdown_connections 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3390969363 wait complete. 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.343+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd66a39aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd66a39aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be65128c0 con 0x558be64d7800 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.366 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd668344700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47340/0 (socket says 172.21.15.191:47340) 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 learned_addr learned my addr 172.21.15.191:0/3804144968 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64feea0 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd668344700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x558be73625d0 tx=0x558be6824e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd661336700 1 -- 172.21.15.191:0/3804144968 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f760 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd661336700 1 -- 172.21.15.191:0/3804144968 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6577e00 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd661336700 1 -- 172.21.15.191:0/3804144968 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be658f4a0 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558be64fe820 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.344+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558be64fe9c0 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd661336700 1 -- 172.21.15.191:0/3804144968 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558be7278dc0 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd661336700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be687ec00 0x558be67bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd661336700 1 -- 172.21.15.191:0/3804144968 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x558be6551340 con 0x558be64d7800 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd667b43700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be687ec00 0x558be67bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.367 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be64d7c00 0x558be67bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.345+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f000 con 0x558be64d7c00 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd668b45700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be64d7c00 0x558be67bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd667b43700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be687ec00 0x558be67bf600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x558be687bd70 tx=0x558be68f4000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd668b45700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be64d7c00 0x558be67bfb80 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 20 realm 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f400 con 0x558be64d7c00 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.368 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f800 con 0x558be64d7c00 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.346+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687fc00 con 0x558be64d7c00 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be687f000 0x558be67c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f400 con 0x558be687f000 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd668344700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be687f000 0x558be67c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.369 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd668344700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be687f000 0x558be67c0100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 20 searching for the correct realm 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.347+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908000 con 0x558be687f000 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd668344700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6908000 0x558be67c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f400 con 0x558be6908000 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd667b43700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6908000 0x558be67c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd667b43700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6908000 0x558be67c0c00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd667b43700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be687f400 0x558be67c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.348+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.370 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.349+0000 7fd668b45700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be687f400 0x558be67c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.349+0000 7fd668b45700 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be687f400 0x558be67c1180 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.349+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.349+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.349+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.349+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908000 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.371 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.350+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.351+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.352+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.373 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.353+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.354+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.374 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f400 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908000 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.355+0000 7fd66a39aa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908400 con 0x558be64d7c00 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908800 con 0x558be64d7c00 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908c00 con 0x558be687f000 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 zone default found 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 4 Realm: () 2024-04-27T04:34:47.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 zonegroup default 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.356+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909000 con 0x558be687f400 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909400 con 0x558be687f000 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908400 con 0x558be687f000 2024-04-27T04:34:47.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 started sync module instance, tier type = 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.357+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908800 con 0x558be6908000 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.358+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.358+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94059355659008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909000 con 0x558be6908000 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.358+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909c00 con 0x558be64d7c00 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.359+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94059355659008] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.359+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.359+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94059355660160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687fc00 con 0x558be64d7c00 2024-04-27T04:34:47.378 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.359+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f800 con 0x558be687f000 2024-04-27T04:34:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.360+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94059355660160] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.361+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.361+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94059355661312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be687f000 2024-04-27T04:34:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.361+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908800 con 0x558be6908000 2024-04-27T04:34:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.362+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94059355661312] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.362+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.362+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94059355662464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909000 con 0x558be6908000 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.362+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908c00 con 0x558be64d7c00 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.363+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.363+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94059355663616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909400 con 0x558be64d7c00 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.363+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908400 con 0x558be687f400 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.363+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94059355662464] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.364+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94059355663616] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.364+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.364+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94059355664768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687e400 con 0x558be687f400 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.364+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910000 con 0x558be687f400 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.365+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94059355664768] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.365+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.365+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94059355665920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910400 con 0x558be687f400 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.365+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910800 con 0x558be687f400 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94059355665920] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94059355667072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910c00 con 0x558be687f400 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=0 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=1 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=2 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=3 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=4 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=5 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=6 2024-04-27T04:34:47.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94059355667072] v31'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 20 add_watcher() i=7 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.366+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911000 con 0x558be64d7c00 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.367+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.367+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94059357110272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911400 con 0x558be64d7c00 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94059357110272] v31'118 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd64f312700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd66a39aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd66a39aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd66a39aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd66a39aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be65783c0 con 0x558be6911c00 2024-04-27T04:34:47.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd667b43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd667b43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47356/0 (socket says 172.21.15.191:47356) 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.368+0000 7fd667b43700 1 -- 172.21.15.191:0/380988924 learned_addr learned my addr 172.21.15.191:0/380988924 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.369+0000 7fd667b43700 1 -- 172.21.15.191:0/380988924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64feb60 con 0x558be6911c00 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.369+0000 7fd667b43700 1 --2- 172.21.15.191:0/380988924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x558be68975f0 tx=0x558be6825f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a160ad95207b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.369+0000 7fd64eb11700 1 -- 172.21.15.191:0/380988924 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be6916160 con 0x558be6911c00 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.369+0000 7fd64eb11700 1 -- 172.21.15.191:0/380988924 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6932000 con 0x558be6911c00 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.369+0000 7fd64eb11700 1 -- 172.21.15.191:0/380988924 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be69162c0 con 0x558be6911c00 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- 172.21.15.191:0/380988924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 msgr2=0x558be67c0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/380988924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x558be68975f0 tx=0x558be6825f80 comp rx=0 tx=0).stop 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- 172.21.15.191:0/380988924 shutdown_connections 2024-04-27T04:34:47.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/380988924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- 172.21.15.191:0/380988924 >> 172.21.15.191:0/380988924 conn(0x558be6911800 msgr2=0x558be65c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- 172.21.15.191:0/380988924 shutdown_connections 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- 172.21.15.191:0/380988924 wait complete. 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 Processor -- start 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- start start 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd66a39aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be6578500 con 0x558be6911c00 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd667b43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd667b43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47358/0 (socket says 172.21.15.191:47358) 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.370+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 learned_addr learned my addr 172.21.15.191:0/517766599 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be64fed00 con 0x558be6911c00 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd667b43700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x558be691c780 tx=0x558be68f4240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd64d30e700 1 -- 172.21.15.191:0/517766599 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be6916420 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd64d30e700 1 -- 172.21.15.191:0/517766599 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558be6932180 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558be64fe340 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd64d30e700 1 -- 172.21.15.191:0/517766599 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558be6916160 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558be64feb60 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.371+0000 7fd64d30e700 1 -- 172.21.15.191:0/517766599 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558be7279b80 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd64d30e700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be6909c00 0x558be67c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd64d30e700 1 -- 172.21.15.191:0/517766599 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x558be65516c0 con 0x558be6911c00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd668b45700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be6909c00 0x558be67c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be687fc00 0x558be69c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687e400 con 0x558be687fc00 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd668344700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be687fc00 0x558be69c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd668b45700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be6909c00 0x558be67c1700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x558be6897e00 tx=0x558be68f52c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.372+0000 7fd668344700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be687fc00 0x558be69c0000 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.373+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'47 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687fc00 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.373+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be6908c00 0x558be69c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.373+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909400 con 0x558be6908c00 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.373+0000 7fd667b43700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be6908c00 0x558be69c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.373+0000 7fd667b43700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be6908c00 0x558be69c0580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.374+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'20 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.374+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be6908400 0x558be69c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.374+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922000 con 0x558be6908400 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.375+0000 7fd668b45700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be6908400 0x558be69c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.375+0000 7fd668b45700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be6908400 0x558be69c0b00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.376+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.376+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922400 con 0x558be687fc00 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.376+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'119 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687fc00 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.377+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6922800 0x558be69c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.377+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922c00 con 0x558be6922800 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.377+0000 7fd668344700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6922800 0x558be69c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.377+0000 7fd668344700 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6922800 0x558be69c1080 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.378+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'41 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.378+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923000 con 0x558be6908400 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.379+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'34 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.379+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923400 con 0x558be687fc00 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.380+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687fc00 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.380+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923800 con 0x558be6908c00 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.381+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.381+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923c00 con 0x558be6922800 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.382+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.382+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922400 con 0x558be6908400 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.383+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.383+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922c00 con 0x558be6922800 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.384+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'42 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.384+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922000 con 0x558be6908c00 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.385+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.385+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910000 con 0x558be6922800 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.386+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.386+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910400 con 0x558be6908c00 2024-04-27T04:34:47.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.387+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'43 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.387+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910800 con 0x558be687fc00 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.388+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'120 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687fc00 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.388+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910c00 con 0x558be6908400 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.389+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'32 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.389+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911000 con 0x558be6922800 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.390+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'43 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.390+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911400 con 0x558be6908400 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.391+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'26 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.391+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687f800 con 0x558be687fc00 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.392+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687fc00 2024-04-27T04:34:47.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.392+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be687e400 con 0x558be687fc00 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.393+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'121 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687fc00 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.393+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.394+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'33 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.394+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923400 con 0x558be6922800 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.395+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'30 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.395+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923c00 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.396+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.396+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922c00 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.396+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'32 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.396+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923800 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.397+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.397+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922000 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.398+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'35 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.398+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908800 con 0x558be6908c00 2024-04-27T04:34:47.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.399+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.399+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909400 con 0x558be6908400 2024-04-27T04:34:47.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.400+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'26 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.400+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910000 con 0x558be6908c00 2024-04-27T04:34:47.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.401+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'30 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.401+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910800 con 0x558be6908400 2024-04-27T04:34:47.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.402+0000 7fd668b45700 1 -- 172.21.15.191:0/517766599 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'34 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be6908400 2024-04-27T04:34:47.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.402+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911000 con 0x558be6908c00 2024-04-27T04:34:47.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.403+0000 7fd667b43700 1 -- 172.21.15.191:0/517766599 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'44 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908c00 2024-04-27T04:34:47.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.403+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923000 con 0x558be6922800 2024-04-27T04:34:47.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.404+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be6922800 2024-04-27T04:34:47.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.405+0000 7fd64ab09700 20 reqs_thread_entry: start 2024-04-27T04:34:47.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.405+0000 7fd66a39aa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd648b05700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:47.409 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd6482e3700 20 reqs_thread_entry: start 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923c00 con 0x558be687f400 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.406+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6922c00 con 0x558be687f400 2024-04-27T04:34:47.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.407+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(63 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.411 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T04:34:47.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.407+0000 7fd64f312700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.407+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94059357110272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6923800 con 0x558be64d7c00 2024-04-27T04:34:47.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.408+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94059357110272] v31'122 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.408+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94059355659008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be6908000 2024-04-27T04:34:47.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.409+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94059355659008] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.409+0000 7fd66a39aa80 20 remove_watcher() i=0 2024-04-27T04:34:47.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.409+0000 7fd66a39aa80 2 removed watcher, disabling cache 2024-04-27T04:34:47.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.409+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94059355660160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910c00 con 0x558be64d7c00 2024-04-27T04:34:47.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.410+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94059355660160] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.410+0000 7fd66a39aa80 20 remove_watcher() i=1 2024-04-27T04:34:47.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.410+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94059355661312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911400 con 0x558be687f000 2024-04-27T04:34:47.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.411+0000 7fd668344700 1 -- 172.21.15.191:0/3804144968 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94059355661312] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901200 con 0x558be687f000 2024-04-27T04:34:47.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.411+0000 7fd66a39aa80 20 remove_watcher() i=2 2024-04-27T04:34:47.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.411+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94059355662464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6908800 con 0x558be6908000 2024-04-27T04:34:47.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.412+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94059355662464] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6901440 con 0x558be6908000 2024-04-27T04:34:47.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.412+0000 7fd66a39aa80 20 remove_watcher() i=3 2024-04-27T04:34:47.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.412+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94059355663616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6909800 con 0x558be64d7c00 2024-04-27T04:34:47.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.413+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94059355663616] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be64d7c00 2024-04-27T04:34:47.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.413+0000 7fd66a39aa80 20 remove_watcher() i=4 2024-04-27T04:34:47.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.413+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94059355664768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910400 con 0x558be687f400 2024-04-27T04:34:47.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.414+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94059355664768] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.414+0000 7fd66a39aa80 20 remove_watcher() i=5 2024-04-27T04:34:47.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.414+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94059355665920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6910000 con 0x558be687f400 2024-04-27T04:34:47.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.415+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94059355665920] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.415+0000 7fd66a39aa80 20 remove_watcher() i=6 2024-04-27T04:34:47.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.415+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94059355667072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558be6911000 con 0x558be687f400 2024-04-27T04:34:47.422 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T04:34:47.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd668b45700 1 -- 172.21.15.191:0/3804144968 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94059355667072] v31'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558be6900b40 con 0x558be687f400 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 20 remove_watcher() i=7 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be687f400 msgr2=0x558be67c1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be687f400 0x558be67c1180 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be64d7c00 msgr2=0x558be67bfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be64d7c00 0x558be67bfb80 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6908000 msgr2=0x558be67c0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6908000 0x558be67c0c00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be687f000 msgr2=0x558be67c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be687f000 0x558be67c0100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be687ec00 msgr2=0x558be67bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be687ec00 0x558be67bf600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x558be687bd70 tx=0x558be68f4000 comp rx=0 tx=0).stop 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd667b43700 1 -- 172.21.15.191:0/3804144968 reap_dead start 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 msgr2=0x558be67beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x558be73625d0 tx=0x558be6824e40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 shutdown_connections 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/3804144968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d7800 0x558be67beb00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 >> 172.21.15.191:0/3804144968 conn(0x558be64d6c00 msgr2=0x558be65c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.416+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 shutdown_connections 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/3804144968 wait complete. 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be6908c00 msgr2=0x558be69c0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558be6908c00 0x558be69c0580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be687fc00 msgr2=0x558be69c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558be687fc00 0x558be69c0000 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6922800 msgr2=0x558be69c1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558be6922800 0x558be69c1080 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be6908400 msgr2=0x558be69c0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558be6908400 0x558be69c0b00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be6909c00 msgr2=0x558be67c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be6909c00 0x558be67c1700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x558be6897e00 tx=0x558be68f52c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 msgr2=0x558be67c0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/517766599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be6911c00 0x558be67c0680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x558be691c780 tx=0x558be68f4240 comp rx=0 tx=0).stop 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd668344700 1 -- 172.21.15.191:0/517766599 reap_dead start 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 shutdown_connections 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 >> 172.21.15.191:0/517766599 conn(0x558be6911800 msgr2=0x558be65c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 shutdown_connections 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/517766599 wait complete. 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be64d7400 msgr2=0x558be67bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be64d7400 0x558be67bf080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x558be72afa10 tx=0x558be6824b40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 msgr2=0x558be67be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x558be72af980 tx=0x558be65c7920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 shutdown_connections 2024-04-27T04:34:47.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558be64d7400 0x558be67bf080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 --2- 172.21.15.191:0/121810765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558be64d6800 0x558be67be580 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 >> 172.21.15.191:0/121810765 conn(0x558be64d6400 msgr2=0x558be65c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.417+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 shutdown_connections 2024-04-27T04:34:47.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.418+0000 7fd66a39aa80 1 -- 172.21.15.191:0/121810765 wait complete. 2024-04-27T04:34:47.427 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-27T04:34:47.427 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-27T04:34:47.428 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-04-27T04:34:47.474 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.474 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.472+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.472+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.500 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f088925ea80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55bbf8d82c00 0x55bbf906a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f088925ea80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bbf90ad500 con 0x55bbf8d82800 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f088925ea80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bbf8e24a00 con 0x55bbf8d82c00 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0887208700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55bbf8d82800 0x55bbf8e6e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60420/0 (socket says 172.21.15.191:60420) 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0887208700 1 -- 172.21.15.191:0/134659933 learned_addr learned my addr 172.21.15.191:0/134659933 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886a07700 1 --2- 172.21.15.191:0/134659933 >> v2:172.21.15.138:3300/0 conn(0x55bbf8d82c00 0x55bbf906a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886206700 1 -- 172.21.15.191:0/134659933 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296133387 0 0) 0x55bbf9b50000 con 0x55bbf8d82800 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886206700 1 -- 172.21.15.191:0/134659933 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55bbf90ad680 con 0x55bbf8d82800 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886a07700 1 -- 172.21.15.191:0/134659933 >> v1:172.21.15.138:6789/0 conn(0x55bbf8d82800 legacy=0x55bbf8e6e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886a07700 1 -- 172.21.15.191:0/134659933 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8dabd40 con 0x55bbf8d82c00 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886a07700 1 --2- 172.21.15.191:0/134659933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82c00 0x55bbf906a580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55bbf9065890 tx=0x55bbf8e73980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e219b1d481845c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886206700 1 -- 172.21.15.191:0/134659933 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b4a0 con 0x55bbf8d82c00 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886206700 1 -- 172.21.15.191:0/134659933 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf90ad500 con 0x55bbf8d82c00 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.473+0000 7f0886206700 1 -- 172.21.15.191:0/134659933 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b600 con 0x55bbf8d82c00 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 -- 172.21.15.191:0/134659933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82c00 msgr2=0x55bbf906a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 --2- 172.21.15.191:0/134659933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82c00 0x55bbf906a580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55bbf9065890 tx=0x55bbf8e73980 comp rx=0 tx=0).stop 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 -- 172.21.15.191:0/134659933 shutdown_connections 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 --2- 172.21.15.191:0/134659933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82c00 0x55bbf906a580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 -- 172.21.15.191:0/134659933 >> 172.21.15.191:0/134659933 conn(0x55bbf8d82400 msgr2=0x55bbf8e6e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 -- 172.21.15.191:0/134659933 shutdown_connections 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.474+0000 7f088925ea80 1 -- 172.21.15.191:0/134659933 wait complete. 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.502 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f088925ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f088925ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf8e24500 con 0x55bbf8d82800 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0887208700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0887208700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47378/0 (socket says 172.21.15.191:47378) 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0887208700 1 -- 172.21.15.191:0/1502406768 learned_addr learned my addr 172.21.15.191:0/1502406768 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0887208700 1 -- 172.21.15.191:0/1502406768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8dabba0 con 0x55bbf8d82800 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0887208700 1 --2- 172.21.15.191:0/1502406768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55bbf9b53770 tx=0x55bbf8e73b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bec273b6a582e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0885a05700 1 -- 172.21.15.191:0/1502406768 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b1e0 con 0x55bbf8d82800 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0885a05700 1 -- 172.21.15.191:0/1502406768 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf90adb00 con 0x55bbf8d82800 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.475+0000 7f0885a05700 1 -- 172.21.15.191:0/1502406768 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b4a0 con 0x55bbf8d82800 2024-04-27T04:34:47.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- 172.21.15.191:0/1502406768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 msgr2=0x55bbf906a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 --2- 172.21.15.191:0/1502406768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55bbf9b53770 tx=0x55bbf8e73b60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- 172.21.15.191:0/1502406768 shutdown_connections 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 --2- 172.21.15.191:0/1502406768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- 172.21.15.191:0/1502406768 >> 172.21.15.191:0/1502406768 conn(0x55bbf8d82400 msgr2=0x55bbf8e6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- 172.21.15.191:0/1502406768 shutdown_connections 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- 172.21.15.191:0/1502406768 wait complete. 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.504 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.476+0000 7f088925ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf8e243c0 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0887208700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0887208700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47384/0 (socket says 172.21.15.191:47384) 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0887208700 1 -- 172.21.15.191:0/386051740 learned_addr learned my addr 172.21.15.191:0/386051740 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0887208700 1 -- 172.21.15.191:0/386051740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8daba00 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0887208700 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55bbf9b53e30 tx=0x55bbf8e73920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0884202700 1 -- 172.21.15.191:0/386051740 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b600 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0884202700 1 -- 172.21.15.191:0/386051740 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf8e23c80 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbf8dab860 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f0884202700 1 -- 172.21.15.191:0/386051740 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b1e0 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.477+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbf8dab6c0 con 0x55bbf8d82800 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.505 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0884202700 1 -- 172.21.15.191:0/386051740 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbf9b24b00 con 0x55bbf8d82800 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f088925ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f088925ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf8dbe640 con 0x55bbf8d83400 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0884202700 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910e000 0x55bbf906b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0884202700 1 -- 172.21.15.191:0/386051740 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55bbf8dfd180 con 0x55bbf8d82800 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887a09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887a09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47390/0 (socket says 172.21.15.191:47390) 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887a09700 1 -- 172.21.15.191:0/1307015655 learned_addr learned my addr 172.21.15.191:0/1307015655 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887208700 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910e000 0x55bbf906b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.506 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887a09700 1 -- 172.21.15.191:0/1307015655 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8daa000 con 0x55bbf8d83400 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887208700 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910e000 0x55bbf906b080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55bbf9c0cf00 tx=0x55bbf90cc240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f0887a09700 1 --2- 172.21.15.191:0/1307015655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55bbf9c0c870 tx=0x55bbf90ccde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93eb04ba059abc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.478+0000 7f08819fd700 1 -- 172.21.15.191:0/1307015655 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b600 con 0x55bbf8d83400 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f08819fd700 1 -- 172.21.15.191:0/1307015655 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf90ac480 con 0x55bbf8d83400 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f08819fd700 1 -- 172.21.15.191:0/1307015655 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b1e0 con 0x55bbf8d83400 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- 172.21.15.191:0/1307015655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 msgr2=0x55bbf906ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 --2- 172.21.15.191:0/1307015655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55bbf9c0c870 tx=0x55bbf90ccde0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- 172.21.15.191:0/1307015655 shutdown_connections 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 --2- 172.21.15.191:0/1307015655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- 172.21.15.191:0/1307015655 >> 172.21.15.191:0/1307015655 conn(0x55bbf8d82c00 msgr2=0x55bbf8e6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- 172.21.15.191:0/1307015655 shutdown_connections 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- 172.21.15.191:0/1307015655 wait complete. 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f088925ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf8dbe8c0 con 0x55bbf8d83400 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f0887a09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.479+0000 7f0887a09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47392/0 (socket says 172.21.15.191:47392) 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 learned_addr learned my addr 172.21.15.191:0/1666764191 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8daaea0 con 0x55bbf8d83400 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f0887a09700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55bbf9c0df20 tx=0x55bbf90ccd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f08801fa700 1 -- 172.21.15.191:0/1666764191 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3ba20 con 0x55bbf8d83400 2024-04-27T04:34:47.508 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f08801fa700 1 -- 172.21.15.191:0/1666764191 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf90ac180 con 0x55bbf8d83400 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbf8daa820 con 0x55bbf8d83400 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f08801fa700 1 -- 172.21.15.191:0/1666764191 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b600 con 0x55bbf8d83400 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.480+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbf8daa9c0 con 0x55bbf8d83400 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f08801fa700 1 -- 172.21.15.191:0/1666764191 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbf9b24dc0 con 0x55bbf8d83400 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f08801fa700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910ec00 0x55bbf906b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f08801fa700 1 -- 172.21.15.191:0/1666764191 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55bbf8dfd340 con 0x55bbf8d83400 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f0886a07700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910ec00 0x55bbf906b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbf8d83800 0x55bbf906bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf8d83c00 con 0x55bbf8d83800 2024-04-27T04:34:47.509 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.481+0000 7f0887a09700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbf8d83800 0x55bbf906bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f0886a07700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910ec00 0x55bbf906b600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55bbf9065890 tx=0x55bbf8e73800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f0887a09700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbf8d83800 0x55bbf906bb80 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 20 realm 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f000 con 0x55bbf8d83800 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f400 con 0x55bbf8d83800 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.482+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f800 con 0x55bbf8d83800 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbf910fc00 0x55bbf906c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f000 con 0x55bbf910fc00 2024-04-27T04:34:47.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f0887208700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbf910fc00 0x55bbf906c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f0887208700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbf910fc00 0x55bbf906c100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 20 searching for the correct realm 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.483+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f400 con 0x55bbf910fc00 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0887208700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbf910f400 0x55bbf906cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f000 con 0x55bbf910f400 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0886a07700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbf910f400 0x55bbf906cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0886a07700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbf910f400 0x55bbf906cc00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.512 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0886a07700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbf910f000 0x55bbf906d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.484+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.485+0000 7f0887a09700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbf910f000 0x55bbf906d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.485+0000 7f0887a09700 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbf910f000 0x55bbf906d180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.485+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.485+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f400 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.486+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f400 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.487+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.488+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.489+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f400 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.490+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f400 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910fc00 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.491+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f400 2024-04-27T04:34:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf8d83c00 con 0x55bbf8d83800 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6400 con 0x55bbf8d83800 2024-04-27T04:34:47.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6800 con 0x55bbf910fc00 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 zone default found 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 4 Realm: () 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 zonegroup default 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.492+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6c00 con 0x55bbf910f000 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7000 con 0x55bbf910fc00 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7400 con 0x55bbf910fc00 2024-04-27T04:34:47.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 started sync module instance, tier type = 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.493+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7800 con 0x55bbf910f400 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.494+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.494+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94265825172224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6800 con 0x55bbf910f400 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.494+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7000 con 0x55bbf8d83800 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.495+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.495+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94265825173376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7400 con 0x55bbf8d83800 2024-04-27T04:34:47.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.495+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6400 con 0x55bbf910fc00 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.495+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94265825172224] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.496+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.496+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94265825174528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6c00 con 0x55bbf910fc00 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.496+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf8d83c00 con 0x55bbf910f400 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.497+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94265825173376] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.497+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94265825174528] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.497+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.497+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94265825175680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf910f800 con 0x55bbf910f400 2024-04-27T04:34:47.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.498+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf8d83800 2024-04-27T04:34:47.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.498+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.498+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94265825176832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7800 con 0x55bbf8d83800 2024-04-27T04:34:47.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.498+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94265825175680] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.499+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6800 con 0x55bbf910f000 2024-04-27T04:34:47.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.499+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.499+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94265825177984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7c00 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.499+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7000 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.500+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94265825176832] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.500+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.500+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94265825179136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7400 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.500+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6000 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.501+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94265825177984] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94265825180288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7800 con 0x55bbf910f000 2024-04-27T04:34:47.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94265825179136] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=0 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=1 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=2 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=3 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=4 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=5 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=6 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94265825180288] v31'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 20 add_watcher() i=7 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6800 con 0x55bbf8d83800 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.504+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94265826746368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7c00 con 0x55bbf8d83800 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.505+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94265826746368] v31'123 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.505+0000 7f086e1d6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.505+0000 7f088925ea80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.505+0000 7f088925ea80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f088925ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f088925ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf8e243c0 con 0x55bbf91d7400 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f0886a07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f0886a07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47402/0 (socket says 172.21.15.191:47402) 2024-04-27T04:34:47.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f0886a07700 1 -- 172.21.15.191:0/112717683 learned_addr learned my addr 172.21.15.191:0/112717683 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f0886a07700 1 -- 172.21.15.191:0/112717683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8daab60 con 0x55bbf91d7400 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f0886a07700 1 --2- 172.21.15.191:0/112717683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55bbf9125440 tx=0x55bbf8e739e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce328ca2f897741d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.506+0000 7f086d9d5700 1 -- 172.21.15.191:0/112717683 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf8e3b760 con 0x55bbf91d7400 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f086d9d5700 1 -- 172.21.15.191:0/112717683 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf91ec000 con 0x55bbf91d7400 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f086d9d5700 1 -- 172.21.15.191:0/112717683 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf91fc000 con 0x55bbf91d7400 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 -- 172.21.15.191:0/112717683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 msgr2=0x55bbf906c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 --2- 172.21.15.191:0/112717683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55bbf9125440 tx=0x55bbf8e739e0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 -- 172.21.15.191:0/112717683 shutdown_connections 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 --2- 172.21.15.191:0/112717683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 -- 172.21.15.191:0/112717683 >> 172.21.15.191:0/112717683 conn(0x55bbf91d7000 msgr2=0x55bbf8e70000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 -- 172.21.15.191:0/112717683 shutdown_connections 2024-04-27T04:34:47.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 -- 172.21.15.191:0/112717683 wait complete. 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.507+0000 7f088925ea80 1 Processor -- start 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f088925ea80 1 -- start start 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f088925ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f088925ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf8e24500 con 0x55bbf91d7400 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f0886a07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f0886a07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47406/0 (socket says 172.21.15.191:47406) 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f0886a07700 1 -- 172.21.15.191:0/3488027560 learned_addr learned my addr 172.21.15.191:0/3488027560 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f0886a07700 1 -- 172.21.15.191:0/3488027560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf8daad00 con 0x55bbf91d7400 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f0886a07700 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55bbf91eb260 tx=0x55bbf8e738c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f086c1d2700 1 -- 172.21.15.191:0/3488027560 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf91fc160 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f086c1d2700 1 -- 172.21.15.191:0/3488027560 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbf91ec180 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbf8daa340 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f086c1d2700 1 -- 172.21.15.191:0/3488027560 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbf91fc2c0 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.508+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbf8dabd40 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.509+0000 7f086c1d2700 1 -- 172.21.15.191:0/3488027560 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbf9b25b80 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.509+0000 7f086c1d2700 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf9276000 0x55bbf906d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.509+0000 7f086c1d2700 1 -- 172.21.15.191:0/3488027560 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55bbf8dfd500 con 0x55bbf91d7400 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.509+0000 7f0887a09700 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf9276000 0x55bbf906d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.509+0000 7f088925ea80 5 note: GC not initialized 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f0887a09700 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf9276000 0x55bbf906d700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55bbf916e570 tx=0x55bbf9205200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f08699cd700 20 reqs_thread_entry: start 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f088925ea80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f088925ea80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f08679c9700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f08671a7700 20 reqs_thread_entry: start 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f088925ea80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.510+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7800 con 0x55bbf910f000 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6800 con 0x55bbf910f400 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.511+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9276400 con 0x55bbf910f000 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:47.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.512+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9276800 con 0x55bbf910f000 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(65 fud.buckets [call] v31'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9276c00 con 0x55bbf910f400 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.513+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9277400 con 0x55bbf910f400 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.514+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v31'21 uv21 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f088925ea80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f088925ea80 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f088925ea80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94265826748672 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9277800 con 0x55bbf910f000 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== watch-notify(notify (1) cookie 94265825179136 notify 133143986189 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x55bbf917dd40 con 0x55bbf910f000 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f08819fd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986189 cookie 94265825179136 notifier 4548 bl.length()=338 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f08819fd700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f08819fd700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f08819fd700 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d6800 con 0x55bbf910f000 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(68 notify.6 [notify cookie 94265826748672 out=8b] v0'0 uv120 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.515+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== watch-notify(notify_complete (2) cookie 94265826748672 notify 133143986189 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bbf923da00 con 0x55bbf910f000 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.516+0000 7f086e1d6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.516+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94265826746368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9277c00 con 0x55bbf8d83800 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.517+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94265826746368] v31'124 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.517+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94265825172224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9276400 con 0x55bbf910f400 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.518+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94265825172224] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.518+0000 7f088925ea80 20 remove_watcher() i=0 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.518+0000 7f088925ea80 2 removed watcher, disabling cache 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.518+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94265825173376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9276800 con 0x55bbf8d83800 2024-04-27T04:34:47.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.519+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94265825173376] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.519+0000 7f088925ea80 20 remove_watcher() i=1 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.519+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94265825174528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9277800 con 0x55bbf910fc00 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.520+0000 7f0887208700 1 -- 172.21.15.191:0/1666764191 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94265825174528] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd200 con 0x55bbf910fc00 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.520+0000 7f088925ea80 20 remove_watcher() i=2 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.520+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94265825175680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9277c00 con 0x55bbf910f400 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.521+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94265825175680] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bd440 con 0x55bbf910f400 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.521+0000 7f088925ea80 20 remove_watcher() i=3 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.521+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94265825176832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9276c00 con 0x55bbf8d83800 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.522+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94265825176832] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf8d83800 2024-04-27T04:34:47.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.522+0000 7f088925ea80 20 remove_watcher() i=4 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.522+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94265825177984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf9277400 con 0x55bbf910f000 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.522+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94265825177984] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.522+0000 7f088925ea80 20 remove_watcher() i=5 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.522+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94265825179136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7c00 con 0x55bbf910f000 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.523+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94265825179136] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.523+0000 7f088925ea80 20 remove_watcher() i=6 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.523+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94265825180288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbf91d7800 con 0x55bbf910f000 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f0887a09700 1 -- 172.21.15.191:0/1666764191 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94265825180288] v31'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbf91bcb40 con 0x55bbf910f000 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 20 remove_watcher() i=7 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbf910f000 msgr2=0x55bbf906d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbf910f000 0x55bbf906d180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbf8d83800 msgr2=0x55bbf906bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbf8d83800 0x55bbf906bb80 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbf910f400 msgr2=0x55bbf906cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbf910f400 0x55bbf906cc00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbf910fc00 msgr2=0x55bbf906c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbf910fc00 0x55bbf906c100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910ec00 msgr2=0x55bbf906b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910ec00 0x55bbf906b600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55bbf9065890 tx=0x55bbf8e73800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 msgr2=0x55bbf906ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.524+0000 7f088925ea80 1 --2- 172.21.15.191:0/1666764191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d83400 0x55bbf906ab00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55bbf9c0df20 tx=0x55bbf90ccd20 comp rx=0 tx=0).stop 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 reap_dead start 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 shutdown_connections 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 >> 172.21.15.191:0/1666764191 conn(0x55bbf8d82c00 msgr2=0x55bbf8e6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f0886a07700 1 -- 172.21.15.191:0/1666764191 reap_dead start 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 shutdown_connections 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/1666764191 wait complete. 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf9276000 msgr2=0x55bbf906d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf9276000 0x55bbf906d700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55bbf916e570 tx=0x55bbf9205200 comp rx=0 tx=0).stop 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 msgr2=0x55bbf906c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55bbf91eb260 tx=0x55bbf8e738c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 shutdown_connections 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf9276000 0x55bbf906d700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/3488027560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf91d7400 0x55bbf906c680 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 >> 172.21.15.191:0/3488027560 conn(0x55bbf91d7000 msgr2=0x55bbf8e70000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 shutdown_connections 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/3488027560 wait complete. 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910e000 msgr2=0x55bbf906b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910e000 0x55bbf906b080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55bbf9c0cf00 tx=0x55bbf90cc240 comp rx=0 tx=0).stop 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 msgr2=0x55bbf906a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55bbf9b53e30 tx=0x55bbf8e73920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 shutdown_connections 2024-04-27T04:34:47.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbf910e000 0x55bbf906b080 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 --2- 172.21.15.191:0/386051740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf8d82800 0x55bbf906a580 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.525+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 >> 172.21.15.191:0/386051740 conn(0x55bbf8d82400 msgr2=0x55bbf8e6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.526+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 shutdown_connections 2024-04-27T04:34:47.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.526+0000 7f088925ea80 1 -- 172.21.15.191:0/386051740 wait complete. 2024-04-27T04:34:47.559 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1'] 2024-04-27T04:34:47.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1'] 2024-04-27T04:34:47.559 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:47.608 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.609 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b058c7a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55636716ac00 0x556367452580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b058c7a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556367495500 con 0x55636716a800 2024-04-27T04:34:47.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b058c7a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55636720ca00 con 0x55636716ac00 2024-04-27T04:34:47.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03070700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55636716ac00 0x556367452580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03871700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55636716a800 0x556367256800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60434/0 (socket says 172.21.15.191:60434) 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03070700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55636716ac00 0x556367452580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47416/0 (socket says 172.21.15.191:47416) 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03871700 1 -- 172.21.15.191:0/763716451 learned_addr learned my addr 172.21.15.191:0/763716451 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03070700 1 -- 172.21.15.191:0/763716451 >> v1:172.21.15.138:6789/0 conn(0x55636716a800 legacy=0x556367256800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03070700 1 -- 172.21.15.191:0/763716451 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367193d40 con 0x55636716ac00 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.607+0000 7f9b03070700 1 --2- 172.21.15.191:0/763716451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716ac00 0x556367452580 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55636744d890 tx=0x55636725b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1129846f34fc078 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b0286f700 1 -- 172.21.15.191:0/763716451 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563672234a0 con 0x55636716ac00 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b0286f700 1 -- 172.21.15.191:0/763716451 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556367495800 con 0x55636716ac00 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b0286f700 1 -- 172.21.15.191:0/763716451 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556367223600 con 0x55636716ac00 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- 172.21.15.191:0/763716451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716ac00 msgr2=0x556367452580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/763716451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716ac00 0x556367452580 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55636744d890 tx=0x55636725b920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- 172.21.15.191:0/763716451 shutdown_connections 2024-04-27T04:34:47.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/763716451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716ac00 0x556367452580 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- 172.21.15.191:0/763716451 >> 172.21.15.191:0/763716451 conn(0x55636716a400 msgr2=0x556367256000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- 172.21.15.191:0/763716451 shutdown_connections 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- 172.21.15.191:0/763716451 wait complete. 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b058c7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636720c500 con 0x55636716a800 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b03871700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b03871700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47430/0 (socket says 172.21.15.191:47430) 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b03871700 1 -- 172.21.15.191:0/3628007145 learned_addr learned my addr 172.21.15.191:0/3628007145 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b03871700 1 -- 172.21.15.191:0/3628007145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367193ba0 con 0x55636716a800 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b03871700 1 --2- 172.21.15.191:0/3628007145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x556367f43e00 tx=0x55636725bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e012e45e7221a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.609+0000 7f9b0206e700 1 -- 172.21.15.191:0/3628007145 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563672231e0 con 0x55636716a800 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b0206e700 1 -- 172.21.15.191:0/3628007145 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556367495b00 con 0x55636716a800 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b0206e700 1 -- 172.21.15.191:0/3628007145 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563672234a0 con 0x55636716a800 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3628007145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 msgr2=0x556367452580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/3628007145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x556367f43e00 tx=0x55636725bb60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3628007145 shutdown_connections 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/3628007145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3628007145 >> 172.21.15.191:0/3628007145 conn(0x55636716a400 msgr2=0x556367257000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3628007145 shutdown_connections 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3628007145 wait complete. 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.610+0000 7f9b058c7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636720c3c0 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b03871700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b03871700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47440/0 (socket says 172.21.15.191:47440) 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b03871700 1 -- 172.21.15.191:0/2346816592 learned_addr learned my addr 172.21.15.191:0/2346816592 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b03871700 1 -- 172.21.15.191:0/2346816592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367193a00 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b03871700 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x556367f43ef0 tx=0x55636725ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b0086b700 1 -- 172.21.15.191:0/2346816592 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556367223600 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b0086b700 1 -- 172.21.15.191:0/2346816592 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55636720bc80 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b0086b700 1 -- 172.21.15.191:0/2346816592 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563672231e0 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556367193860 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.611+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563671936c0 con 0x55636716a800 2024-04-27T04:34:47.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b0086b700 1 -- 172.21.15.191:0/2346816592 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556367f0cb00 con 0x55636716a800 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b058c7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b058c7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563671a6640 con 0x55636716b400 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b0086b700 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500000 0x556367453080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b0086b700 1 -- 172.21.15.191:0/2346816592 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5563671e5180 con 0x55636716a800 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b04072700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b03871700 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500000 0x556367453080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b04072700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47452/0 (socket says 172.21.15.191:47452) 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b04072700 1 -- 172.21.15.191:0/1759114014 learned_addr learned my addr 172.21.15.191:0/1759114014 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b04072700 1 -- 172.21.15.191:0/1759114014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367192000 con 0x55636716b400 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b03871700 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500000 0x556367453080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x556367fecf90 tx=0x5563674ba240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.612+0000 7f9b04072700 1 --2- 172.21.15.191:0/1759114014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x556367fec2a0 tx=0x5563674bb0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66befe1f6f41453e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9afe066700 1 -- 172.21.15.191:0/1759114014 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556367223600 con 0x55636716b400 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9afe066700 1 -- 172.21.15.191:0/1759114014 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556367494300 con 0x55636716b400 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9afe066700 1 -- 172.21.15.191:0/1759114014 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563672231e0 con 0x55636716b400 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- 172.21.15.191:0/1759114014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 msgr2=0x556367452b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/1759114014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x556367fec2a0 tx=0x5563674bb0e0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- 172.21.15.191:0/1759114014 shutdown_connections 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/1759114014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- 172.21.15.191:0/1759114014 >> 172.21.15.191:0/1759114014 conn(0x55636716ac00 msgr2=0x556367257800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- 172.21.15.191:0/1759114014 shutdown_connections 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- 172.21.15.191:0/1759114014 wait complete. 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.613+0000 7f9b058c7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563671a68c0 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b04072700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b04072700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47460/0 (socket says 172.21.15.191:47460) 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 learned_addr learned my addr 172.21.15.191:0/79523331 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367192ea0 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b04072700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x556367fec7e0 tx=0x5563674bacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9afc863700 1 -- 172.21.15.191:0/79523331 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556367223a20 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9afc863700 1 -- 172.21.15.191:0/79523331 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55636720bc80 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556367192820 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9afc863700 1 -- 172.21.15.191:0/79523331 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556367223600 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.614+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563671929c0 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9afc863700 1 -- 172.21.15.191:0/79523331 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556367f0cdc0 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9afc863700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500c00 0x556367453600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9afc863700 1 -- 172.21.15.191:0/79523331 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5563671e5340 con 0x55636716b400 2024-04-27T04:34:47.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9b03070700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500c00 0x556367453600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55636716b800 0x556367453b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636716bc00 con 0x55636716b800 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9b04072700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55636716b800 0x556367453b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.615+0000 7f9b03070700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500c00 0x556367453600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55636744d890 tx=0x55636725b800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b04072700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55636716b800 0x556367453b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 realm 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501000 con 0x55636716b800 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501400 con 0x55636716b800 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501800 con 0x55636716b800 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.616+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556367501c00 0x556367454100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501000 con 0x556367501c00 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b03871700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556367501c00 0x556367454100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b03871700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556367501c00 0x556367454100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b058c7a80 20 searching for the correct realm 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501400 con 0x556367501c00 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.617+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03871700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556367501400 0x556367454c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501000 con 0x556367501400 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03070700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556367501400 0x556367454c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03070700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556367501400 0x556367454c00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03070700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556367501000 0x556367455180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.618+0000 7f9b04072700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556367501000 0x556367455180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556367501000 0x556367455180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.619+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501400 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.620+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501400 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.621+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.622+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501400 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.623+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.624+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501400 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501c00 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.625+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501400 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636716bc00 con 0x55636716b800 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2400 con 0x55636716b800 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2800 con 0x556367501c00 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 zone default found 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 4 Realm: () 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 zonegroup default 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.626+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2c00 con 0x556367501000 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3000 con 0x556367501c00 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3400 con 0x556367501c00 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 started sync module instance, tier type = 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.627+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3800 con 0x556367501400 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.628+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.628+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93885422672640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2800 con 0x556367501400 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.628+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3000 con 0x55636716b800 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.629+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93885422672640] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.630+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.630+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93885422673792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3400 con 0x55636716b800 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.630+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2400 con 0x556367501c00 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.631+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.631+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93885422674944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2c00 con 0x556367501c00 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.631+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636716bc00 con 0x556367501400 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.631+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93885422673792] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.632+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.632+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93885422674944] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.632+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93885422676096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556367501800 con 0x556367501400 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.632+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x55636716b800 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.633+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93885422676096] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.633+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.633+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93885422677248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3800 con 0x55636716b800 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.633+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2800 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.634+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.634+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93885422677248] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.634+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93885422678400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3c00 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.634+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3000 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.635+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.635+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93885422679552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3400 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.635+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2000 con 0x556367501000 2024-04-27T04:34:47.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.635+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93885422678400] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93885422680704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3800 con 0x556367501000 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=0 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=1 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=2 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=3 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=4 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=5 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93885422679552] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.636+0000 7f9b058c7a80 20 add_watcher() i=6 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.637+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93885422680704] v31'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.638+0000 7f9b058c7a80 20 add_watcher() i=7 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.638+0000 7f9b058c7a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.638+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2800 con 0x55636716b800 2024-04-27T04:34:47.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.638+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.638+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93885424271360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3c00 con 0x55636716b800 2024-04-27T04:34:47.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93885424271360] v31'125 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9aea83f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b058c7a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b058c7a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b058c7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.639+0000 7f9b058c7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636720c3c0 con 0x5563675c3400 2024-04-27T04:34:47.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9b03070700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9b03070700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47462/0 (socket says 172.21.15.191:47462) 2024-04-27T04:34:47.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9b03070700 1 -- 172.21.15.191:0/2794196296 learned_addr learned my addr 172.21.15.191:0/2794196296 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9b03070700 1 -- 172.21.15.191:0/2794196296 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367192b60 con 0x5563675c3400 2024-04-27T04:34:47.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9b03070700 1 --2- 172.21.15.191:0/2794196296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55636756ff20 tx=0x55636725b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b8615169198427 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9aea03e700 1 -- 172.21.15.191:0/2794196296 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556367223760 con 0x5563675c3400 2024-04-27T04:34:47.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9aea03e700 1 -- 172.21.15.191:0/2794196296 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563675de000 con 0x5563675c3400 2024-04-27T04:34:47.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.640+0000 7f9aea03e700 1 -- 172.21.15.191:0/2794196296 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563675e0000 con 0x5563675c3400 2024-04-27T04:34:47.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2794196296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 msgr2=0x556367454680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/2794196296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55636756ff20 tx=0x55636725b980 comp rx=0 tx=0).stop 2024-04-27T04:34:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2794196296 shutdown_connections 2024-04-27T04:34:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/2794196296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2794196296 >> 172.21.15.191:0/2794196296 conn(0x5563675c3000 msgr2=0x556367258000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2794196296 shutdown_connections 2024-04-27T04:34:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2794196296 wait complete. 2024-04-27T04:34:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 Processor -- start 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- start start 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b058c7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636720c500 con 0x5563675c3400 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b03070700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b03070700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47464/0 (socket says 172.21.15.191:47464) 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.641+0000 7f9b03070700 1 -- 172.21.15.191:0/3603188082 learned_addr learned my addr 172.21.15.191:0/3603188082 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9b03070700 1 -- 172.21.15.191:0/3603188082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556367192d00 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9b03070700 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x5563675d81e0 tx=0x55636725b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9ae883b700 1 -- 172.21.15.191:0/3603188082 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563675e0160 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9ae883b700 1 -- 172.21.15.191:0/3603188082 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563675de180 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556367192340 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9ae883b700 1 -- 172.21.15.191:0/3603188082 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5563675e02c0 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.642+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556367193d40 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.643+0000 7f9ae883b700 1 -- 172.21.15.191:0/3603188082 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556367f0db80 con 0x5563675c3400 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.643+0000 7f9ae883b700 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55636765c000 0x556367455700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.643+0000 7f9ae883b700 1 -- 172.21.15.191:0/3603188082 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5563671e5500 con 0x5563675c3400 2024-04-27T04:34:47.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.643+0000 7f9b04072700 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55636765c000 0x556367455700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.643+0000 7f9b058c7a80 5 note: GC not initialized 2024-04-27T04:34:47.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.643+0000 7f9b04072700 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55636765c000 0x556367455700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55636756fc80 tx=0x5563675e1320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9ae6036700 20 reqs_thread_entry: start 2024-04-27T04:34:47.691 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9ae4032700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:17 2024 2024-04-27T04:34:47.692 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9ae3810700 20 reqs_thread_entry: start 2024-04-27T04:34:47.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:47.692 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3800 con 0x55636716b800 2024-04-27T04:34:47.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:47.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:47.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:47.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:47.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.644+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2800 con 0x556367501400 2024-04-27T04:34:47.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:47.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:34:47.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:47.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765c400 con 0x556367501000 2024-04-27T04:34:47.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:47.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:47.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T04:34:47.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.645+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765c800 con 0x556367501000 2024-04-27T04:34:47.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.646+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(65 fud.buckets [call] v31'3 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.646+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765d000 con 0x556367501000 2024-04-27T04:34:47.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(66 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v31'3 uv3 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 10 moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:47.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [notify cookie 93885424273664 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765d800 con 0x556367501400 2024-04-27T04:34:47.698 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== watch-notify(notify (1) cookie 93885422676096 notify 133143986190 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x556367625ba0 con 0x556367501400 2024-04-27T04:34:47.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9afe066700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986190 cookie 93885422676096 notifier 4556 bl.length()=725 2024-04-27T04:34:47.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 notify.3 [notify cookie 93885424273664 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9afe066700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:47.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9afe066700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:47.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9afe066700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:47.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9afe066700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3c00 con 0x556367501400 2024-04-27T04:34:47.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.3 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== watch-notify(notify_complete (2) cookie 93885424273664 notify 133143986190 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x556367625d40 con 0x556367501400 2024-04-27T04:34:47.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_sources= new_sources= 2024-04-27T04:34:47.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential sources added=[] removed=[] 2024-04-27T04:34:47.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_dests= new_dests= 2024-04-27T04:34:47.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential dests added=[] removed=[] 2024-04-27T04:34:47.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.648+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765dc00 con 0x556367501400 2024-04-27T04:34:47.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.650+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(69 foo.buckets [call] v31'22 uv22 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.650+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765d800 con 0x556367501400 2024-04-27T04:34:47.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v31'23 uv23 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b058c7a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T04:34:47.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b058c7a80 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b058c7a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:34:47.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93885424273664 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765c400 con 0x556367501000 2024-04-27T04:34:47.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== watch-notify(notify (1) cookie 93885422679552 notify 133143986190 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x556367565d40 con 0x556367501000 2024-04-27T04:34:47.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9afe066700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986190 cookie 93885422679552 notifier 4556 bl.length()=314 2024-04-27T04:34:47.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9afe066700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T04:34:47.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9afe066700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:47.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9afe066700 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3800 con 0x556367501000 2024-04-27T04:34:47.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(71 notify.6 [notify cookie 93885424273664 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.651+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== watch-notify(notify_complete (2) cookie 93885424273664 notify 133143986190 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x556367621a00 con 0x556367501000 2024-04-27T04:34:47.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.652+0000 7f9aea83f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.652+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93885424271360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765c800 con 0x55636716b800 2024-04-27T04:34:47.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.653+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 93885424271360] v31'126 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.653+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93885422672640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765d400 con 0x556367501400 2024-04-27T04:34:47.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.654+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 93885422672640] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.654+0000 7f9b058c7a80 20 remove_watcher() i=0 2024-04-27T04:34:47.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.654+0000 7f9b058c7a80 2 removed watcher, disabling cache 2024-04-27T04:34:47.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.654+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93885422673792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765cc00 con 0x55636716b800 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.655+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 93885422673792] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.655+0000 7f9b058c7a80 20 remove_watcher() i=1 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.655+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93885422674944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636765d000 con 0x556367501c00 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.656+0000 7f9b03871700 1 -- 172.21.15.191:0/79523331 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 93885422674944] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3200 con 0x556367501c00 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.656+0000 7f9b058c7a80 20 remove_watcher() i=2 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.656+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93885422676096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c2800 con 0x556367501400 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.657+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 93885422676096] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b3440 con 0x556367501400 2024-04-27T04:34:47.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.657+0000 7f9b058c7a80 20 remove_watcher() i=3 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.657+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93885422677248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563675c3c00 con 0x55636716b800 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.658+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 93885422677248] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x55636716b800 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.658+0000 7f9b058c7a80 20 remove_watcher() i=4 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.658+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93885422678400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55636716bc00 con 0x556367501000 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.659+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 93885422678400] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.659+0000 7f9b058c7a80 20 remove_watcher() i=5 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.659+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93885422679552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563671d6000 con 0x556367501000 2024-04-27T04:34:47.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.660+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 93885422679552] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.660+0000 7f9b058c7a80 20 remove_watcher() i=6 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.660+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93885422680704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563671d6400 con 0x556367501000 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b04072700 1 -- 172.21.15.191:0/79523331 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 93885422680704] v31'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5563675b2b40 con 0x556367501000 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 20 remove_watcher() i=7 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556367501000 msgr2=0x556367455180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556367501000 0x556367455180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55636716b800 msgr2=0x556367453b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55636716b800 0x556367453b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556367501400 msgr2=0x556367454c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556367501400 0x556367454c00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556367501c00 msgr2=0x556367454100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556367501c00 0x556367454100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500c00 msgr2=0x556367453600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500c00 0x556367453600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55636744d890 tx=0x55636725b800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 msgr2=0x556367452b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x556367fec7e0 tx=0x5563674bacc0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b03070700 1 -- 172.21.15.191:0/79523331 reap_dead start 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 shutdown_connections 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/79523331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716b400 0x556367452b00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 >> 172.21.15.191:0/79523331 conn(0x55636716ac00 msgr2=0x556367257800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 shutdown_connections 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.661+0000 7f9b058c7a80 1 -- 172.21.15.191:0/79523331 wait complete. 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55636765c000 msgr2=0x556367455700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55636765c000 0x556367455700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55636756fc80 tx=0x5563675e1320 comp rx=0 tx=0).stop 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 msgr2=0x556367454680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x5563675d81e0 tx=0x55636725b8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 shutdown_connections 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55636765c000 0x556367455700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/3603188082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563675c3400 0x556367454680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 >> 172.21.15.191:0/3603188082 conn(0x5563675c3000 msgr2=0x556367258000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 shutdown_connections 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/3603188082 wait complete. 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500000 msgr2=0x556367453080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500000 0x556367453080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x556367fecf90 tx=0x5563674ba240 comp rx=0 tx=0).stop 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 msgr2=0x556367452580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x556367f43ef0 tx=0x55636725ba40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 shutdown_connections 2024-04-27T04:34:47.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556367500000 0x556367453080 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 --2- 172.21.15.191:0/2346816592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55636716a800 0x556367452580 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 >> 172.21.15.191:0/2346816592 conn(0x55636716a400 msgr2=0x556367257000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 shutdown_connections 2024-04-27T04:34:47.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.662+0000 7f9b058c7a80 1 -- 172.21.15.191:0/2346816592 wait complete. 2024-04-27T04:34:47.711 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T04:34:47.711 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T04:34:47.711 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T04:34:47.756 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.757 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.754+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b8c49fa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55bbb7f42c00 0x55bbb822a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b8c49fa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bbb826d500 con 0x55bbb7f42800 2024-04-27T04:34:47.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b8c49fa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bbb7fe4a00 con 0x55bbb7f42c00 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b89c48700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55bbb7f42c00 0x55bbb822a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b8a449700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55bbb7f42800 0x55bbb802e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60450/0 (socket says 172.21.15.191:60450) 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b8a449700 1 -- 172.21.15.191:0/1561961317 learned_addr learned my addr 172.21.15.191:0/1561961317 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b89c48700 1 -- 172.21.15.191:0/1561961317 >> v1:172.21.15.138:6789/0 conn(0x55bbb7f42800 legacy=0x55bbb802e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b89c48700 1 -- 172.21.15.191:0/1561961317 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6bd40 con 0x55bbb7f42c00 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b89c48700 1 --2- 172.21.15.191:0/1561961317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42c00 0x55bbb822a580 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55bbb8225890 tx=0x55bbb8033920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b453dd830d5d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.755+0000 7f3b89447700 1 -- 172.21.15.191:0/1561961317 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb4a0 con 0x55bbb7f42c00 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b89447700 1 -- 172.21.15.191:0/1561961317 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb826d800 con 0x55bbb7f42c00 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b89447700 1 -- 172.21.15.191:0/1561961317 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb600 con 0x55bbb7f42c00 2024-04-27T04:34:47.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/1561961317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42c00 msgr2=0x55bbb822a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/1561961317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42c00 0x55bbb822a580 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55bbb8225890 tx=0x55bbb8033920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/1561961317 shutdown_connections 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/1561961317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42c00 0x55bbb822a580 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/1561961317 >> 172.21.15.191:0/1561961317 conn(0x55bbb7f42400 msgr2=0x55bbb802e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/1561961317 shutdown_connections 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.756+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/1561961317 wait complete. 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8c49fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8c49fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb7fe4500 con 0x55bbb7f42800 2024-04-27T04:34:47.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8a449700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8a449700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47480/0 (socket says 172.21.15.191:47480) 2024-04-27T04:34:47.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8a449700 1 -- 172.21.15.191:0/2931517568 learned_addr learned my addr 172.21.15.191:0/2931517568 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8a449700 1 -- 172.21.15.191:0/2931517568 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6bba0 con 0x55bbb7f42800 2024-04-27T04:34:47.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b8a449700 1 --2- 172.21.15.191:0/2931517568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55bbb8d1b800 tx=0x55bbb8033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da6765160ad6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b88c46700 1 -- 172.21.15.191:0/2931517568 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb4a0 con 0x55bbb7f42800 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b88c46700 1 -- 172.21.15.191:0/2931517568 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb826db00 con 0x55bbb7f42800 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.757+0000 7f3b88c46700 1 -- 172.21.15.191:0/2931517568 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb600 con 0x55bbb7f42800 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2931517568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 msgr2=0x55bbb822a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/2931517568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55bbb8d1b800 tx=0x55bbb8033b60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2931517568 shutdown_connections 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/2931517568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2931517568 >> 172.21.15.191:0/2931517568 conn(0x55bbb7f42400 msgr2=0x55bbb802f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2931517568 shutdown_connections 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2931517568 wait complete. 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8c49fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb7fe43c0 con 0x55bbb7f42800 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8a449700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8a449700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47488/0 (socket says 172.21.15.191:47488) 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.758+0000 7f3b8a449700 1 -- 172.21.15.191:0/2576557270 learned_addr learned my addr 172.21.15.191:0/2576557270 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b8a449700 1 -- 172.21.15.191:0/2576557270 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6ba00 con 0x55bbb7f42800 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b8a449700 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55bbb8d1ba10 tx=0x55bbb8033a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b87443700 1 -- 172.21.15.191:0/2576557270 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb8c0 con 0x55bbb7f42800 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b87443700 1 -- 172.21.15.191:0/2576557270 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb7fe3c80 con 0x55bbb7f42800 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b87443700 1 -- 172.21.15.191:0/2576557270 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb4a0 con 0x55bbb7f42800 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbb7f6b860 con 0x55bbb7f42800 2024-04-27T04:34:47.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbb7f6b6c0 con 0x55bbb7f42800 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.759+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8c49fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8c49fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb7f7e640 con 0x55bbb7f43400 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b87443700 1 -- 172.21.15.191:0/2576557270 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbb8ce4b00 con 0x55bbb7f42800 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8ac4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8ac4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47498/0 (socket says 172.21.15.191:47498) 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4000907990 learned_addr learned my addr 172.21.15.191:0/4000907990 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b87443700 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8000 0x55bbb822b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b87443700 1 -- 172.21.15.191:0/2576557270 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55bbb7fbd180 con 0x55bbb7f42800 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8a449700 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8000 0x55bbb822b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4000907990 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6a000 con 0x55bbb7f43400 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/4000907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55bbb82d6060 tx=0x55bbb8290de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eb96076fe1a85a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b8a449700 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8000 0x55bbb822b080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55bbb8dc0f90 tx=0x55bbb8290240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b84c3e700 1 -- 172.21.15.191:0/4000907990 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb8c0 con 0x55bbb7f43400 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b84c3e700 1 -- 172.21.15.191:0/4000907990 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb826c300 con 0x55bbb7f43400 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.760+0000 7f3b84c3e700 1 -- 172.21.15.191:0/4000907990 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb4a0 con 0x55bbb7f43400 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4000907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 msgr2=0x55bbb822ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4000907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55bbb82d6060 tx=0x55bbb8290de0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4000907990 shutdown_connections 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4000907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4000907990 >> 172.21.15.191:0/4000907990 conn(0x55bbb7f42c00 msgr2=0x55bbb802f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4000907990 shutdown_connections 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4000907990 wait complete. 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.761+0000 7f3b8c49fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb7f7e8c0 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8ac4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8ac4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47512/0 (socket says 172.21.15.191:47512) 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 learned_addr learned my addr 172.21.15.191:0/3938939179 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6aea0 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55bbb8dc0240 tx=0x55bbb8290840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8343b700 1 -- 172.21.15.191:0/3938939179 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffbb80 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8343b700 1 -- 172.21.15.191:0/3938939179 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb7fe3c80 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbb7f6a820 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8343b700 1 -- 172.21.15.191:0/3938939179 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb8c0 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.762+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbb7f6a9c0 con 0x55bbb7f43400 2024-04-27T04:34:47.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8343b700 1 -- 172.21.15.191:0/3938939179 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbb8ce4dc0 con 0x55bbb7f43400 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8343b700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8c00 0x55bbb822b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8343b700 1 -- 172.21.15.191:0/3938939179 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55bbb7fbd340 con 0x55bbb7f43400 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b89c48700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8c00 0x55bbb822b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb7f43800 0x55bbb822bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb7f43c00 con 0x55bbb7f43800 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb7f43800 0x55bbb822bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.763+0000 7f3b89c48700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8c00 0x55bbb822b600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55bbb8225890 tx=0x55bbb8033800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb7f43800 0x55bbb822bb80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 realm 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9000 con 0x55bbb7f43800 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9400 con 0x55bbb7f43800 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.764+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9800 con 0x55bbb7f43800 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb82d9c00 0x55bbb822c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9000 con 0x55bbb82d9c00 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8a449700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb82d9c00 0x55bbb822c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8a449700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb82d9c00 0x55bbb822c100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 20 searching for the correct realm 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.765+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9400 con 0x55bbb82d9c00 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b8a449700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb82d9400 0x55bbb822cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9000 con 0x55bbb82d9400 2024-04-27T04:34:47.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b89c48700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb82d9400 0x55bbb822cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b89c48700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb82d9400 0x55bbb822cc00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b89c48700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb82d9000 0x55bbb822d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.766+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.767+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb82d9000 0x55bbb822d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.767+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb82d9000 0x55bbb822d180 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.767+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.767+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.767+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.767+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9400 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.768+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9400 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.769+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.770+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.771+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9400 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.772+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9400 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9c00 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.773+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9400 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb7f43c00 con 0x55bbb7f43800 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398400 con 0x55bbb7f43800 2024-04-27T04:34:47.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.774+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398800 con 0x55bbb82d9c00 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 zone default found 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 4 Realm: () 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 zonegroup default 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398c00 con 0x55bbb82d9000 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399000 con 0x55bbb82d9c00 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399400 con 0x55bbb82d9c00 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 started sync module instance, tier type = 2024-04-27T04:34:47.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.775+0000 7f3b8c49fa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.776+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb82d9400 2024-04-27T04:34:47.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.777+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.777+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94264736488192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398800 con 0x55bbb82d9400 2024-04-27T04:34:47.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.777+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399000 con 0x55bbb7f43800 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.778+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94264736488192] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.778+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.778+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94264736489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399400 con 0x55bbb7f43800 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.778+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398400 con 0x55bbb82d9c00 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.779+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.779+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94264736489344] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.779+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94264736490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398c00 con 0x55bbb82d9c00 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.779+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb7f43c00 con 0x55bbb82d9400 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.780+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94264736490496] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.780+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.780+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94264736491648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb82d9800 con 0x55bbb82d9400 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.780+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb7f43800 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.781+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94264736491648] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.782+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.782+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94264736492800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb7f43800 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.782+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398800 con 0x55bbb82d9000 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.783+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.783+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94264736493952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399c00 con 0x55bbb82d9000 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.783+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399000 con 0x55bbb82d9000 2024-04-27T04:34:47.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.783+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94264736492800] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.783+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94264736493952] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.784+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.784+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94264736495104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399400 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.784+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398000 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.784+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94264736495104] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94264736496256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb82d9000 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=0 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=1 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=2 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=3 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=4 2024-04-27T04:34:47.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=5 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=6 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94264736496256] v31'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 20 add_watcher() i=7 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.785+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398800 con 0x55bbb7f43800 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.786+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.786+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94264738078720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399c00 con 0x55bbb7f43800 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94264738078720] v31'127 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b71417700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8c49fa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8c49fa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8c49fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.787+0000 7f3b8c49fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb7fe43c0 con 0x55bbb8399400 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b89c48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b89c48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47522/0 (socket says 172.21.15.191:47522) 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b89c48700 1 -- 172.21.15.191:0/403981818 learned_addr learned my addr 172.21.15.191:0/403981818 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b89c48700 1 -- 172.21.15.191:0/403981818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6ab60 con 0x55bbb8399400 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b89c48700 1 --2- 172.21.15.191:0/403981818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55bbb83ae000 tx=0x55bbb8033980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2369f3ecf4737122 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b70c16700 1 -- 172.21.15.191:0/403981818 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb7ffb760 con 0x55bbb8399400 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b70c16700 1 -- 172.21.15.191:0/403981818 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb83ba000 con 0x55bbb8399400 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.788+0000 7f3b70c16700 1 -- 172.21.15.191:0/403981818 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb83bc000 con 0x55bbb8399400 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/403981818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 msgr2=0x55bbb822c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/403981818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55bbb83ae000 tx=0x55bbb8033980 comp rx=0 tx=0).stop 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/403981818 shutdown_connections 2024-04-27T04:34:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/403981818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/403981818 >> 172.21.15.191:0/403981818 conn(0x55bbb8399000 msgr2=0x55bbb8030000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/403981818 shutdown_connections 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/403981818 wait complete. 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 Processor -- start 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- start start 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.789+0000 7f3b8c49fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb7fe4500 con 0x55bbb8399400 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b89c48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b89c48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47526/0 (socket says 172.21.15.191:47526) 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 learned_addr learned my addr 172.21.15.191:0/4236552870 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb7f6ad00 con 0x55bbb8399400 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b89c48700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55bbb83ae240 tx=0x55bbb80338c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b6f413700 1 -- 172.21.15.191:0/4236552870 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb83bc160 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b6f413700 1 -- 172.21.15.191:0/4236552870 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbb83ba180 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbb7f6a340 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b6f413700 1 -- 172.21.15.191:0/4236552870 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbb83bc2c0 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.790+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbb7f6bd40 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b6f413700 1 -- 172.21.15.191:0/4236552870 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbb8ce5b80 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b6f413700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb8434000 0x55bbb822d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b6f413700 1 -- 172.21.15.191:0/4236552870 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55bbb7fbd500 con 0x55bbb8399400 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb8434000 0x55bbb822d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb8398000 0x55bbb844a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb8398000 2024-04-27T04:34:47.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b8a449700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb8398000 0x55bbb844a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb8434000 0x55bbb822d700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55bbb833bcb0 tx=0x55bbb83c5260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.791+0000 7f3b8a449700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb8398000 0x55bbb844a000 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.792+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'48 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8398000 2024-04-27T04:34:47.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.792+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb8398800 0x55bbb844a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.792+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8434400 con 0x55bbb8398800 2024-04-27T04:34:47.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.793+0000 7f3b89c48700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb8398800 0x55bbb844a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.793+0000 7f3b89c48700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb8398800 0x55bbb844a580 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.794+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'21 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.794+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb8434800 0x55bbb844ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.794+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8434c00 con 0x55bbb8434800 2024-04-27T04:34:47.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.794+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb8434800 0x55bbb844ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.794+0000 7f3b8ac4a700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb8434800 0x55bbb844ab00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.795+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.795+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435000 con 0x55bbb8398000 2024-04-27T04:34:47.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.796+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'128 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8398000 2024-04-27T04:34:47.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.796+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb8435400 0x55bbb844b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.796+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435800 con 0x55bbb8435400 2024-04-27T04:34:47.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.796+0000 7f3b8a449700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb8435400 0x55bbb844b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.796+0000 7f3b8a449700 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb8435400 0x55bbb844b080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.797+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'44 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.797+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435c00 con 0x55bbb8434800 2024-04-27T04:34:47.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.798+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'36 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.798+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435000 con 0x55bbb8398000 2024-04-27T04:34:47.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.799+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8398000 2024-04-27T04:34:47.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.800+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398400 con 0x55bbb8398800 2024-04-27T04:34:47.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.800+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.800+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398c00 con 0x55bbb8435400 2024-04-27T04:34:47.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.801+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.801+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb8434800 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.802+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.802+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e000 con 0x55bbb8435400 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.803+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'45 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.803+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e400 con 0x55bbb8398800 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.804+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.804+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e800 con 0x55bbb8435400 2024-04-27T04:34:47.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.805+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.805+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847ec00 con 0x55bbb8398800 2024-04-27T04:34:47.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.806+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'45 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.806+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f000 con 0x55bbb8398000 2024-04-27T04:34:47.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.807+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'129 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8398000 2024-04-27T04:34:47.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.807+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f400 con 0x55bbb8434800 2024-04-27T04:34:47.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.808+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'35 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.808+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f800 con 0x55bbb8435400 2024-04-27T04:34:47.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.809+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'46 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.809+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847fc00 con 0x55bbb8434800 2024-04-27T04:34:47.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.810+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'27 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.810+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e000 con 0x55bbb8398000 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.811+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8398000 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.811+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e800 con 0x55bbb8398000 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.812+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'130 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8398000 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.812+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e400 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.813+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'36 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.813+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398c00 con 0x55bbb8435400 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.814+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'31 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.814+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398400 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.815+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.815+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435800 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.816+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'33 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.816+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435000 con 0x55bbb8434800 2024-04-27T04:34:47.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.817+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.817+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8434400 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.818+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'37 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.818+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb7f43c00 con 0x55bbb8398800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.819+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.819+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f400 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.820+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'27 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.820+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847fc00 con 0x55bbb8398800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.821+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'32 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.821+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e400 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.821+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/4236552870 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'37 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb8434800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.821+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399c00 con 0x55bbb8398800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.822+0000 7f3b89c48700 1 -- 172.21.15.191:0/4236552870 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'46 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb8398800 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.822+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb8435400 2024-04-27T04:34:47.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.823+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb8435400 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.824+0000 7f3b6cc0e700 20 reqs_thread_entry: start 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.824+0000 7f3b8c49fa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.824+0000 7f3b8c49fa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.824+0000 7f3b6ac0a700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b6a3e8700 20 reqs_thread_entry: start 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435c00 con 0x55bbb82d9000 2024-04-27T04:34:47.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398c00 con 0x55bbb82d9000 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(63 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+15 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.825+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8399800 con 0x55bbb82d9000 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.826+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(64 p5YnriCv1nAtykxBrupQ [delete] v31'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.826+0000 7f3b8c49fa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.826+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94264738082176 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f000 con 0x55bbb82d9c00 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94264736490496 notify 133143986179 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55bbb83fbba0 con 0x55bbb82d9c00 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b84c3e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 94264736490496 notifier 4564 bl.length()=170 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(65 notify.2 [notify cookie 94264738082176 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b84c3e700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f400 con 0x55bbb82d9c00 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 94264738082176 notify 133143986179 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bbb83fbd40 con 0x55bbb82d9c00 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8c49fa80 10 removing email index: 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8c49fa80 10 removing user buckets index 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.827+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847f800 con 0x55bbb82d9000 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(67 fud.buckets [delete] v31'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8c49fa80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94264738082176 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e000 con 0x55bbb82d9000 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== watch-notify(notify (1) cookie 94264736495104 notify 133143986191 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55bbb83f9a00 con 0x55bbb82d9000 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b84c3e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 94264736495104 notifier 4564 bl.length()=160 2024-04-27T04:34:47.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(68 notify.6 [notify cookie 94264738082176 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b84c3e700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e400 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== watch-notify(notify_complete (2) cookie 94264738082176 notify 133143986191 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bbb83f7040 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8c49fa80 10 removing user index: fud 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.828+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb847e800 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.829+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(70 fud [call,call,delete] v31'2 uv2 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.829+0000 7f3b8c49fa80 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.829+0000 7f3b8c49fa80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.829+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [notify cookie 94264738082176 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398400 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.830+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== watch-notify(notify (1) cookie 94264736493952 notify 133143986192 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55bbb83f7040 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.830+0000 7f3b84c3e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94264736493952 notifier 4564 bl.length()=152 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.830+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(71 notify.5 [notify cookie 94264738082176 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.830+0000 7f3b84c3e700 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8508000 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.830+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== watch-notify(notify_complete (2) cookie 94264738082176 notify 133143986192 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bbb83f71e0 con 0x55bbb82d9000 2024-04-27T04:34:47.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.830+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== osd_op_reply(72 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.831+0000 7f3b71417700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.831+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94264738078720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8398c00 con 0x55bbb7f43800 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.832+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94264738078720] v31'131 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.832+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94264736488192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435800 con 0x55bbb82d9400 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.833+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94264736488192] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.833+0000 7f3b8c49fa80 20 remove_watcher() i=0 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.833+0000 7f3b8c49fa80 2 removed watcher, disabling cache 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.833+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94264736489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435000 con 0x55bbb7f43800 2024-04-27T04:34:47.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.834+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94264736489344] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.834+0000 7f3b8c49fa80 20 remove_watcher() i=1 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.834+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94264736490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8434400 con 0x55bbb82d9c00 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.835+0000 7f3b8a449700 1 -- 172.21.15.191:0/3938939179 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94264736490496] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389200 con 0x55bbb82d9c00 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.835+0000 7f3b8c49fa80 20 remove_watcher() i=2 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.835+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94264736491648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8435c00 con 0x55bbb82d9400 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.836+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94264736491648] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8389440 con 0x55bbb82d9400 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.836+0000 7f3b8c49fa80 20 remove_watcher() i=3 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.836+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94264736492800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8508000 con 0x55bbb7f43800 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.837+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94264736492800] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb7f43800 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.837+0000 7f3b8c49fa80 20 remove_watcher() i=4 2024-04-27T04:34:47.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.837+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94264736493952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8508c00 con 0x55bbb82d9000 2024-04-27T04:34:47.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.838+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94264736493952] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.838+0000 7f3b8c49fa80 20 remove_watcher() i=5 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.838+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94264736495104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8509000 con 0x55bbb82d9000 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.839+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94264736495104] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.839+0000 7f3b8c49fa80 20 remove_watcher() i=6 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.839+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94264736496256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bbb8509400 con 0x55bbb82d9000 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8ac4a700 1 -- 172.21.15.191:0/3938939179 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94264736496256] v31'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbb8388b40 con 0x55bbb82d9000 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 20 remove_watcher() i=7 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb82d9000 msgr2=0x55bbb822d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb82d9000 0x55bbb822d180 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb7f43800 msgr2=0x55bbb822bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb7f43800 0x55bbb822bb80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb82d9400 msgr2=0x55bbb822cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb82d9400 0x55bbb822cc00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb82d9c00 msgr2=0x55bbb822c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb82d9c00 0x55bbb822c100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8c00 msgr2=0x55bbb822b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8c00 0x55bbb822b600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55bbb8225890 tx=0x55bbb8033800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 msgr2=0x55bbb822ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/3938939179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f43400 0x55bbb822ab00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55bbb8dc0240 tx=0x55bbb8290840 comp rx=0 tx=0).stop 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 reap_dead start 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b89c48700 1 -- 172.21.15.191:0/3938939179 reap_dead start 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 shutdown_connections 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 >> 172.21.15.191:0/3938939179 conn(0x55bbb7f42c00 msgr2=0x55bbb802f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 shutdown_connections 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/3938939179 wait complete. 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb8398800 msgr2=0x55bbb844a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbb8398800 0x55bbb844a580 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb8398000 msgr2=0x55bbb844a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbb8398000 0x55bbb844a000 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb8435400 msgr2=0x55bbb844b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbb8435400 0x55bbb844b080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb8434800 msgr2=0x55bbb844ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbb8434800 0x55bbb844ab00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb8434000 msgr2=0x55bbb822d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb8434000 0x55bbb822d700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55bbb833bcb0 tx=0x55bbb83c5260 comp rx=0 tx=0).stop 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 msgr2=0x55bbb822c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/4236552870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb8399400 0x55bbb822c680 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55bbb83ae240 tx=0x55bbb80338c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 reap_dead start 2024-04-27T04:34:47.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.840+0000 7f3b8a449700 1 -- 172.21.15.191:0/4236552870 reap_dead start 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 shutdown_connections 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 >> 172.21.15.191:0/4236552870 conn(0x55bbb8399000 msgr2=0x55bbb8030000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 shutdown_connections 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/4236552870 wait complete. 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8000 msgr2=0x55bbb822b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8000 0x55bbb822b080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55bbb8dc0f90 tx=0x55bbb8290240 comp rx=0 tx=0).stop 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 msgr2=0x55bbb822a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55bbb8d1ba10 tx=0x55bbb8033a40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 shutdown_connections 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbb82d8000 0x55bbb822b080 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 --2- 172.21.15.191:0/2576557270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbb7f42800 0x55bbb822a580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 >> 172.21.15.191:0/2576557270 conn(0x55bbb7f42400 msgr2=0x55bbb802f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 shutdown_connections 2024-04-27T04:34:47.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.841+0000 7f3b8c49fa80 1 -- 172.21.15.191:0/2576557270 wait complete. 2024-04-27T04:34:47.856 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T04:34:47.856 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T04:34:47.856 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-27T04:34:47.911 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:47.911 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:47.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a737b8a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5640f335cc00 0x5640f3644580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a737b8a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5640f3687500 con 0x5640f335c800 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a737b8a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5640f33fea00 con 0x5640f335cc00 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a70f61700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5640f335cc00 0x5640f3644580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a71762700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5640f335c800 0x5640f3448800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60464/0 (socket says 172.21.15.191:60464) 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.909+0000 7f1a71762700 1 -- 172.21.15.191:0/445472633 learned_addr learned my addr 172.21.15.191:0/445472633 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a70f61700 1 -- 172.21.15.191:0/445472633 >> v1:172.21.15.138:6789/0 conn(0x5640f335c800 legacy=0x5640f3448800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a70f61700 1 -- 172.21.15.191:0/445472633 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3385d40 con 0x5640f335cc00 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a70f61700 1 --2- 172.21.15.191:0/445472633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335cc00 0x5640f3644580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5640f363f890 tx=0x5640f344d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427dc3af137e6863 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a70760700 1 -- 172.21.15.191:0/445472633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f34154a0 con 0x5640f335cc00 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a70760700 1 -- 172.21.15.191:0/445472633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f3687800 con 0x5640f335cc00 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a70760700 1 -- 172.21.15.191:0/445472633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f3415600 con 0x5640f335cc00 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a737b8a80 1 -- 172.21.15.191:0/445472633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335cc00 msgr2=0x5640f3644580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/445472633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335cc00 0x5640f3644580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5640f363f890 tx=0x5640f344d920 comp rx=0 tx=0).stop 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a737b8a80 1 -- 172.21.15.191:0/445472633 shutdown_connections 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/445472633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335cc00 0x5640f3644580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.910+0000 7f1a737b8a80 1 -- 172.21.15.191:0/445472633 >> 172.21.15.191:0/445472633 conn(0x5640f335c400 msgr2=0x5640f3448000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.911+0000 7f1a737b8a80 1 -- 172.21.15.191:0/445472633 shutdown_connections 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.911+0000 7f1a737b8a80 1 -- 172.21.15.191:0/445472633 wait complete. 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.911+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.911+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.911+0000 7f1a737b8a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.911+0000 7f1a737b8a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f33fe500 con 0x5640f335c800 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a71762700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a71762700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47546/0 (socket says 172.21.15.191:47546) 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a71762700 1 -- 172.21.15.191:0/4028301696 learned_addr learned my addr 172.21.15.191:0/4028301696 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a71762700 1 -- 172.21.15.191:0/4028301696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3385ba0 con 0x5640f335c800 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a71762700 1 --2- 172.21.15.191:0/4028301696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5640f363ea50 tx=0x5640f344db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0da0071ad41fe7f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a6ff5f700 1 -- 172.21.15.191:0/4028301696 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f34151e0 con 0x5640f335c800 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a6ff5f700 1 -- 172.21.15.191:0/4028301696 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f3687b00 con 0x5640f335c800 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a6ff5f700 1 -- 172.21.15.191:0/4028301696 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f34154a0 con 0x5640f335c800 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.912+0000 7f1a737b8a80 1 -- 172.21.15.191:0/4028301696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 msgr2=0x5640f3644580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/4028301696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5640f363ea50 tx=0x5640f344db60 comp rx=0 tx=0).stop 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 -- 172.21.15.191:0/4028301696 shutdown_connections 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/4028301696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 -- 172.21.15.191:0/4028301696 >> 172.21.15.191:0/4028301696 conn(0x5640f335c400 msgr2=0x5640f3449000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 -- 172.21.15.191:0/4028301696 shutdown_connections 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 -- 172.21.15.191:0/4028301696 wait complete. 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a737b8a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f33fe3c0 con 0x5640f335c800 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a71762700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a71762700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47558/0 (socket says 172.21.15.191:47558) 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a71762700 1 -- 172.21.15.191:0/1750615005 learned_addr learned my addr 172.21.15.191:0/1750615005 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.913+0000 7f1a71762700 1 -- 172.21.15.191:0/1750615005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3385a00 con 0x5640f335c800 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a71762700 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5640f363ffb0 tx=0x5640f344da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a6e75c700 1 -- 172.21.15.191:0/1750615005 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f3415600 con 0x5640f335c800 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a6e75c700 1 -- 172.21.15.191:0/1750615005 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f33fdc80 con 0x5640f335c800 2024-04-27T04:34:47.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a6e75c700 1 -- 172.21.15.191:0/1750615005 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f34151e0 con 0x5640f335c800 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5640f3385860 con 0x5640f335c800 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640f33856c0 con 0x5640f335c800 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a6e75c700 1 -- 172.21.15.191:0/1750615005 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5640f40feb00 con 0x5640f335c800 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a737b8a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.914+0000 7f1a737b8a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f3398640 con 0x5640f335d400 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a6e75c700 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ec000 0x5640f3645080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a6e75c700 1 -- 172.21.15.191:0/1750615005 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5640f33d7180 con 0x5640f335c800 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71f63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71f63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47570/0 (socket says 172.21.15.191:47570) 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71f63700 1 -- 172.21.15.191:0/293946248 learned_addr learned my addr 172.21.15.191:0/293946248 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71762700 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ec000 0x5640f3645080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71f63700 1 -- 172.21.15.191:0/293946248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3384000 con 0x5640f335d400 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71762700 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ec000 0x5640f3645080 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5640f41dcf90 tx=0x5640f36a8240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a71f63700 1 --2- 172.21.15.191:0/293946248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5640f36f47b0 tx=0x5640f36a90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf0c477fe25a3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a6bf57700 1 -- 172.21.15.191:0/293946248 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f3415600 con 0x5640f335d400 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a6bf57700 1 -- 172.21.15.191:0/293946248 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f3686300 con 0x5640f335d400 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a6bf57700 1 -- 172.21.15.191:0/293946248 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f34151e0 con 0x5640f335d400 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a737b8a80 1 -- 172.21.15.191:0/293946248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 msgr2=0x5640f3644b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.915+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/293946248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5640f36f47b0 tx=0x5640f36a90e0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 -- 172.21.15.191:0/293946248 shutdown_connections 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/293946248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 -- 172.21.15.191:0/293946248 >> 172.21.15.191:0/293946248 conn(0x5640f335cc00 msgr2=0x5640f3449800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 -- 172.21.15.191:0/293946248 shutdown_connections 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 -- 172.21.15.191:0/293946248 wait complete. 2024-04-27T04:34:47.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a737b8a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f33988c0 con 0x5640f335d400 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a71f63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a71f63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47586/0 (socket says 172.21.15.191:47586) 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 learned_addr learned my addr 172.21.15.191:0/1824473403 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.916+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3384ea0 con 0x5640f335d400 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a71f63700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5640f36f4990 tx=0x5640f36a8d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a6a754700 1 -- 172.21.15.191:0/1824473403 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f3415a20 con 0x5640f335d400 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a6a754700 1 -- 172.21.15.191:0/1824473403 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f33fdc80 con 0x5640f335d400 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5640f3384820 con 0x5640f335d400 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a6a754700 1 -- 172.21.15.191:0/1824473403 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f3415600 con 0x5640f335d400 2024-04-27T04:34:47.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640f33849c0 con 0x5640f335d400 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.917+0000 7f1a6a754700 1 -- 172.21.15.191:0/1824473403 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5640f40fedc0 con 0x5640f335d400 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a6a754700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ecc00 0x5640f3645600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a6a754700 1 -- 172.21.15.191:0/1824473403 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5640f33d7340 con 0x5640f335d400 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a70f61700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ecc00 0x5640f3645600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5640f335d800 0x5640f3645b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f335dc00 con 0x5640f335d800 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a71f63700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5640f335d800 0x5640f3645b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a70f61700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ecc00 0x5640f3645600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5640f363f890 tx=0x5640f344d800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a71f63700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5640f335d800 0x5640f3645b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 20 realm 2024-04-27T04:34:47.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.918+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed000 con 0x5640f335d800 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed400 con 0x5640f335d800 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed800 con 0x5640f335d800 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5640f36edc00 0x5640f3646100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed000 con 0x5640f36edc00 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a71762700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5640f36edc00 0x5640f3646100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.919+0000 7f1a71762700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5640f36edc00 0x5640f3646100 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a737b8a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a737b8a80 20 searching for the correct realm 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed400 con 0x5640f36edc00 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a71762700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5640f36ed400 0x5640f3646c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed000 con 0x5640f36ed400 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a70f61700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5640f36ed400 0x5640f3646c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.920+0000 7f1a70f61700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5640f36ed400 0x5640f3646c00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a70f61700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5640f36ed000 0x5640f3647180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a71f63700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5640f36ed000 0x5640f3647180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a71f63700 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5640f36ed000 0x5640f3647180 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.921+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed400 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.922+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.923+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed400 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.924+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.925+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed400 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.926+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.955 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed400 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36edc00 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.927+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed400 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a737b8a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a737b8a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:47.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a737b8a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a737b8a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.928+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f335dc00 con 0x5640f335d800 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4400 con 0x5640f335d800 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4800 con 0x5640f36edc00 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 zone default found 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 4 Realm: () 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 zonegroup default 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4c00 con 0x5640f36ed000 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.929+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5000 con 0x5640f36edc00 2024-04-27T04:34:47.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5400 con 0x5640f36edc00 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 20 started sync module instance, tier type = 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.930+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5800 con 0x5640f36ed400 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.931+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.931+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94836961295104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4800 con 0x5640f36ed400 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.931+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5000 con 0x5640f335d800 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.932+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94836961295104] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.932+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.932+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94836961296256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5400 con 0x5640f335d800 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.932+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4400 con 0x5640f36edc00 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.933+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.933+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94836961297408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4c00 con 0x5640f36edc00 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.933+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f335dc00 con 0x5640f36ed400 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.933+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94836961296256] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.934+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94836961297408] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.934+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.934+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94836961298560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f36ed800 con 0x5640f36ed400 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.934+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f335d800 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.935+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94836961298560] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.935+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.935+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94836961299712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5800 con 0x5640f335d800 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.935+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4800 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.936+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94836961299712] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.936+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.936+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94836961300864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5c00 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.936+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5000 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.937+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94836961300864] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.937+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.937+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94836961302016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5400 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.937+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4000 con 0x5640f36ed000 2024-04-27T04:34:47.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.938+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94836961302016] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94836961303168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5800 con 0x5640f36ed000 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=0 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=1 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=2 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=3 2024-04-27T04:34:47.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=4 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=5 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=6 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94836961303168] v31'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 20 add_watcher() i=7 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.940+0000 7f1a737b8a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.941+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4800 con 0x5640f335d800 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.941+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.941+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94836962893824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5c00 con 0x5640f335d800 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94836962893824] v31'132 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a58730700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a737b8a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a737b8a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a737b8a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a737b8a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f33fe3c0 con 0x5640f37b5400 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a70f61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a70f61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47596/0 (socket says 172.21.15.191:47596) 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a70f61700 1 -- 172.21.15.191:0/474264143 learned_addr learned my addr 172.21.15.191:0/474264143 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a70f61700 1 -- 172.21.15.191:0/474264143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3384b60 con 0x5640f37b5400 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a70f61700 1 --2- 172.21.15.191:0/474264143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5640f37ca060 tx=0x5640f344d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9aaea182136cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a57f2f700 1 -- 172.21.15.191:0/474264143 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f3415760 con 0x5640f37b5400 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a57f2f700 1 -- 172.21.15.191:0/474264143 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f37d6000 con 0x5640f37b5400 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.943+0000 7f1a57f2f700 1 -- 172.21.15.191:0/474264143 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f37d8000 con 0x5640f37b5400 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- 172.21.15.191:0/474264143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 msgr2=0x5640f3646680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/474264143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5640f37ca060 tx=0x5640f344d980 comp rx=0 tx=0).stop 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- 172.21.15.191:0/474264143 shutdown_connections 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/474264143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- 172.21.15.191:0/474264143 >> 172.21.15.191:0/474264143 conn(0x5640f37b5000 msgr2=0x5640f344a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- 172.21.15.191:0/474264143 shutdown_connections 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- 172.21.15.191:0/474264143 wait complete. 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 Processor -- start 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- start start 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a737b8a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f33fe500 con 0x5640f37b5400 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a70f61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a70f61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47598/0 (socket says 172.21.15.191:47598) 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.944+0000 7f1a70f61700 1 -- 172.21.15.191:0/569958843 learned_addr learned my addr 172.21.15.191:0/569958843 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:47.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a70f61700 1 -- 172.21.15.191:0/569958843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f3384d00 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a70f61700 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5640f37ca270 tx=0x5640f344d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a5672c700 1 -- 172.21.15.191:0/569958843 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f37d8160 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a5672c700 1 -- 172.21.15.191:0/569958843 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640f37d6180 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5640f3384340 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a5672c700 1 -- 172.21.15.191:0/569958843 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5640f37d82c0 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.945+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640f3385d40 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.946+0000 7f1a5672c700 1 -- 172.21.15.191:0/569958843 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5640f40ffb80 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.946+0000 7f1a5672c700 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f3850000 0x5640f3647700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.946+0000 7f1a5672c700 1 -- 172.21.15.191:0/569958843 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5640f33d7500 con 0x5640f37b5400 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.946+0000 7f1a71f63700 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f3850000 0x5640f3647700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:47.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.946+0000 7f1a737b8a80 5 note: GC not initialized 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.946+0000 7f1a71f63700 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f3850000 0x5640f3647700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5640f375bd40 tx=0x5640f37d9320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a53f27700 20 reqs_thread_entry: start 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a737b8a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a737b8a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a51f23700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a51701700 20 reqs_thread_entry: start 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a737b8a80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b5800 con 0x5640f36ed000 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.947+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 246+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f37b4800 con 0x5640f36ed000 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.948+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~385 in=385b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3850400 con 0x5640f36ed000 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.949+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~385] v31'5 uv5 ondisk = 0) v8 ==== 288+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.949+0000 7f1a737b8a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-27T04:34:47.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.949+0000 7f1a737b8a80 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.949+0000 7f1a737b8a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.949+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94836962896128 in=588b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3850800 con 0x5640f36ed400 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== watch-notify(notify (1) cookie 94836961298560 notify 133143986191 ret 0) v3 ==== 618+0+0 (crc 0 0 0) 0x5640f3819ba0 con 0x5640f36ed400 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.3 [notify cookie 94836962896128 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a6bf57700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 94836961298560 notifier 4572 bl.length()=576 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a6bf57700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a6bf57700 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a6bf57700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f38be000 con 0x5640f36ed400 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify_complete (2) cookie 94836962896128 notify 133143986191 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5640f3819d40 con 0x5640f36ed400 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.950+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3850c00 con 0x5640f36ed000 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.952+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v31'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.952+0000 7f1a737b8a80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.952+0000 7f1a737b8a80 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.952+0000 7f1a737b8a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.952+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94836962896128 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851000 con 0x5640f36edc00 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94836961297408 notify 133143986180 ret 0) v3 ==== 234+0+0 (crc 0 0 0) 0x5640f36a7d40 con 0x5640f36edc00 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a6bf57700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94836961297408 notifier 4572 bl.length()=192 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a6bf57700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a6bf57700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(68 notify.2 [notify cookie 94836962896128 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a6bf57700 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f38be400 con 0x5640f36edc00 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.953+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 94836962896128 notify 133143986180 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5640f3817a00 con 0x5640f36edc00 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "testx$tenanteduser", 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "tenanted-user", 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "", 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 1, 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "testx$tenanteduser", 2024-04-27T04:34:47.979 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:47.980 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:47.981 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:47.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.954+0000 7f1a58730700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:47.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.954+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94836962893824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851400 con 0x5640f335d800 2024-04-27T04:34:47.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.955+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94836962893824] v31'133 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.955+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94836961295104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851800 con 0x5640f36ed400 2024-04-27T04:34:47.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.956+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94836961295104] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.956+0000 7f1a737b8a80 20 remove_watcher() i=0 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.956+0000 7f1a737b8a80 2 removed watcher, disabling cache 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.956+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94836961296256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851c00 con 0x5640f335d800 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.957+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94836961296256] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.957+0000 7f1a737b8a80 20 remove_watcher() i=1 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.957+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94836961297408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3850400 con 0x5640f36edc00 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.958+0000 7f1a71762700 1 -- 172.21.15.191:0/1824473403 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94836961297408] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9200 con 0x5640f36edc00 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.958+0000 7f1a737b8a80 20 remove_watcher() i=2 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.958+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94836961298560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3850c00 con 0x5640f36ed400 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.959+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94836961298560] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a9440 con 0x5640f36ed400 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.959+0000 7f1a737b8a80 20 remove_watcher() i=3 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.959+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94836961299712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851400 con 0x5640f335d800 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.960+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94836961299712] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f335d800 2024-04-27T04:34:47.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.960+0000 7f1a737b8a80 20 remove_watcher() i=4 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.960+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94836961300864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851c00 con 0x5640f36ed000 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.961+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94836961300864] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.961+0000 7f1a737b8a80 20 remove_watcher() i=5 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.961+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94836961302016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3850800 con 0x5640f36ed000 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.962+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94836961302016] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.962+0000 7f1a737b8a80 20 remove_watcher() i=6 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.962+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94836961303168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5640f3851800 con 0x5640f36ed000 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a71f63700 1 -- 172.21.15.191:0/1824473403 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94836961303168] v31'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5640f37a8b40 con 0x5640f36ed000 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 20 remove_watcher() i=7 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5640f36ed000 msgr2=0x5640f3647180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5640f36ed000 0x5640f3647180 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5640f335d800 msgr2=0x5640f3645b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5640f335d800 0x5640f3645b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5640f36ed400 msgr2=0x5640f3646c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5640f36ed400 0x5640f3646c00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5640f36edc00 msgr2=0x5640f3646100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5640f36edc00 0x5640f3646100 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ecc00 msgr2=0x5640f3645600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ecc00 0x5640f3645600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5640f363f890 tx=0x5640f344d800 comp rx=0 tx=0).stop 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 msgr2=0x5640f3644b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1824473403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335d400 0x5640f3644b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5640f36f4990 tx=0x5640f36a8d20 comp rx=0 tx=0).stop 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 reap_dead start 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a70f61700 1 -- 172.21.15.191:0/1824473403 reap_dead start 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 shutdown_connections 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 >> 172.21.15.191:0/1824473403 conn(0x5640f335cc00 msgr2=0x5640f3449800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 shutdown_connections 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.963+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1824473403 wait complete. 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f3850000 msgr2=0x5640f3647700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f3850000 0x5640f3647700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5640f375bd40 tx=0x5640f37d9320 comp rx=0 tx=0).stop 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 msgr2=0x5640f3646680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5640f37ca270 tx=0x5640f344d8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 shutdown_connections 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f3850000 0x5640f3647700 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/569958843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f37b5400 0x5640f3646680 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 >> 172.21.15.191:0/569958843 conn(0x5640f37b5000 msgr2=0x5640f344a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 shutdown_connections 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/569958843 wait complete. 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ec000 msgr2=0x5640f3645080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ec000 0x5640f3645080 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5640f41dcf90 tx=0x5640f36a8240 comp rx=0 tx=0).stop 2024-04-27T04:34:47.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 msgr2=0x5640f3644580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5640f363ffb0 tx=0x5640f344da40 comp rx=0 tx=0).stop 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 shutdown_connections 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5640f36ec000 0x5640f3645080 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 --2- 172.21.15.191:0/1750615005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640f335c800 0x5640f3644580 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 >> 172.21.15.191:0/1750615005 conn(0x5640f335c400 msgr2=0x5640f3449000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 shutdown_connections 2024-04-27T04:34:47.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:47.964+0000 7f1a737b8a80 1 -- 172.21.15.191:0/1750615005 wait complete. 2024-04-27T04:34:47.992 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:47.992 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T04:34:47.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T04:34:47.993 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.039+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a1b48da80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a1b48da80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ffbc461500 con 0x55ffbc136c00 2024-04-27T04:34:48.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a1b48da80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ffbc1d8a00 con 0x55ffbc136800 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a19437700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a18c36700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55ffbc136c00 0x55ffbc222800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60480/0 (socket says 172.21.15.191:60480) 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a18c36700 1 -- 172.21.15.191:0/2723176987 learned_addr learned my addr 172.21.15.191:0/2723176987 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a18435700 1 -- 172.21.15.191:0/2723176987 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248767165 0 0) 0x55ffbcf04000 con 0x55ffbc136c00 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a18435700 1 -- 172.21.15.191:0/2723176987 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55ffbc461680 con 0x55ffbc136c00 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a19437700 1 -- 172.21.15.191:0/2723176987 >> v1:172.21.15.138:6789/0 conn(0x55ffbc136c00 legacy=0x55ffbc222800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.040+0000 7f9a19437700 1 -- 172.21.15.191:0/2723176987 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15fd40 con 0x55ffbc136800 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a19437700 1 --2- 172.21.15.191:0/2723176987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4199e0 tx=0x55ffbc227980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba18a6d868a66ec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a18435700 1 -- 172.21.15.191:0/2723176987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef4a0 con 0x55ffbc136800 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a18435700 1 -- 172.21.15.191:0/2723176987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc461500 con 0x55ffbc136800 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a18435700 1 -- 172.21.15.191:0/2723176987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef600 con 0x55ffbc136800 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 -- 172.21.15.191:0/2723176987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 msgr2=0x55ffbc41e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/2723176987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4199e0 tx=0x55ffbc227980 comp rx=0 tx=0).stop 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 -- 172.21.15.191:0/2723176987 shutdown_connections 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/2723176987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 -- 172.21.15.191:0/2723176987 >> 172.21.15.191:0/2723176987 conn(0x55ffbc136400 msgr2=0x55ffbc222000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 -- 172.21.15.191:0/2723176987 shutdown_connections 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.041+0000 7f9a1b48da80 1 -- 172.21.15.191:0/2723176987 wait complete. 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a1b48da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a1b48da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbc1d8500 con 0x55ffbc136800 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47624/0 (socket says 172.21.15.191:47624) 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.042+0000 7f9a19437700 1 -- 172.21.15.191:0/3289658112 learned_addr learned my addr 172.21.15.191:0/3289658112 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a19437700 1 -- 172.21.15.191:0/3289658112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15fba0 con 0x55ffbc136800 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a19437700 1 --2- 172.21.15.191:0/3289658112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf07800 tx=0x55ffbc227800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5132c59a7774cb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a17c34700 1 -- 172.21.15.191:0/3289658112 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef760 con 0x55ffbc136800 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a17c34700 1 -- 172.21.15.191:0/3289658112 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc461980 con 0x55ffbc136800 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a17c34700 1 -- 172.21.15.191:0/3289658112 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef1e0 con 0x55ffbc136800 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3289658112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 msgr2=0x55ffbc41e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3289658112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf07800 tx=0x55ffbc227800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3289658112 shutdown_connections 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3289658112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3289658112 >> 172.21.15.191:0/3289658112 conn(0x55ffbc136400 msgr2=0x55ffbc223000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3289658112 shutdown_connections 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.043+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3289658112 wait complete. 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a1b48da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a1b48da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbc1d83c0 con 0x55ffbc136800 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47632/0 (socket says 172.21.15.191:47632) 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a19437700 1 -- 172.21.15.191:0/253519168 learned_addr learned my addr 172.21.15.191:0/253519168 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a19437700 1 -- 172.21.15.191:0/253519168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15fa00 con 0x55ffbc136800 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a19437700 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf07ec0 tx=0x55ffbc227980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a16431700 1 -- 172.21.15.191:0/253519168 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef4a0 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a16431700 1 -- 172.21.15.191:0/253519168 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc461b00 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ffbc15f860 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a16431700 1 -- 172.21.15.191:0/253519168 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef600 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.044+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbc15f6c0 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a16431700 1 -- 172.21.15.191:0/253519168 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ffbced8b00 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a16431700 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc137400 0x55ffbc41f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a16431700 1 -- 172.21.15.191:0/253519168 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ffbc1b1180 con 0x55ffbc136800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a1b48da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a1b48da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbc172640 con 0x55ffbc137800 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a19c38700 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc137400 0x55ffbc41f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47646/0 (socket says 172.21.15.191:47646) 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.045+0000 7f9a19437700 1 -- 172.21.15.191:0/3089626050 learned_addr learned my addr 172.21.15.191:0/3089626050 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a19437700 1 -- 172.21.15.191:0/3089626050 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15e000 con 0x55ffbc137800 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a19c38700 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc137400 0x55ffbc41f080 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf07fb0 tx=0x55ffbc48cb40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a19437700 1 --2- 172.21.15.191:0/3089626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ffbcfb8f90 tx=0x55ffbc48cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf764d6e4209c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3089626050 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef4a0 con 0x55ffbc137800 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3089626050 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc461b00 con 0x55ffbc137800 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3089626050 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef600 con 0x55ffbc137800 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3089626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 msgr2=0x55ffbc41eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3089626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ffbcfb8f90 tx=0x55ffbc48cc60 comp rx=0 tx=0).stop 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3089626050 shutdown_connections 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3089626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3089626050 >> 172.21.15.191:0/3089626050 conn(0x55ffbc136c00 msgr2=0x55ffbc223800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3089626050 shutdown_connections 2024-04-27T04:34:48.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.046+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3089626050 wait complete. 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a1b48da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a1b48da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbc1728c0 con 0x55ffbc137800 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a19437700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47648/0 (socket says 172.21.15.191:47648) 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 learned_addr learned my addr 172.21.15.191:0/3041619860 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15eea0 con 0x55ffbc137800 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a19437700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4c69c0 tx=0x55ffbc48ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a12429700 1 -- 172.21.15.191:0/3041619860 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef760 con 0x55ffbc137800 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a12429700 1 -- 172.21.15.191:0/3041619860 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc1d7e00 con 0x55ffbc137800 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ffbc15e820 con 0x55ffbc137800 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a12429700 1 -- 172.21.15.191:0/3041619860 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc1ef1e0 con 0x55ffbc137800 2024-04-27T04:34:48.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.047+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbc15e9c0 con 0x55ffbc137800 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a12429700 1 -- 172.21.15.191:0/3041619860 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ffbced8dc0 con 0x55ffbc137800 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a12429700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc4e8c00 0x55ffbc41f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a12429700 1 -- 172.21.15.191:0/3041619860 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ffbc1b1340 con 0x55ffbc137800 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a18c36700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc4e8c00 0x55ffbc41f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ffbc137c00 0x55ffbc41fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.048+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9000 con 0x55ffbc137c00 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a19c38700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ffbc137c00 0x55ffbc41fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a18c36700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc4e8c00 0x55ffbc41f600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf06030 tx=0x55ffbc550000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a19c38700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ffbc137c00 0x55ffbc41fb80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 20 realm 2024-04-27T04:34:48.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9400 con 0x55ffbc137c00 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.049+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9800 con 0x55ffbc137c00 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9c00 con 0x55ffbc137c00 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.084 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ffbc4e9000 0x55ffbc420100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9400 con 0x55ffbc4e9000 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a19437700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ffbc4e9000 0x55ffbc420100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.050+0000 7f9a19437700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ffbc4e9000 0x55ffbc420100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a1b48da80 20 searching for the correct realm 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562000 con 0x55ffbc4e9000 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a19437700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ffbc562000 0x55ffbc420c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9400 con 0x55ffbc562000 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a18c36700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ffbc562000 0x55ffbc420c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.051+0000 7f9a18c36700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ffbc562000 0x55ffbc420c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a18c36700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ffbc4e9400 0x55ffbc421180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a19c38700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ffbc4e9400 0x55ffbc421180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a19c38700 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ffbc4e9400 0x55ffbc421180 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.052+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.086 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.053+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc562000 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.054+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc562000 2024-04-27T04:34:48.087 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.055+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.056+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc562000 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.057+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.058+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.058+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.058+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.058+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.058+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.058+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc562000 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9400 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc137c00 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.059+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc562000 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:48.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562400 con 0x55ffbc137c00 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562800 con 0x55ffbc137c00 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562c00 con 0x55ffbc4e9000 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:48.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 zone default found 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 4 Realm: () 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 zonegroup default 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.060+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563000 con 0x55ffbc4e9400 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563400 con 0x55ffbc4e9000 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562400 con 0x55ffbc4e9000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 started sync module instance, tier type = 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.061+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562800 con 0x55ffbc562000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.062+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.062+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94556863421184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563000 con 0x55ffbc562000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.062+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563c00 con 0x55ffbc137c00 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.063+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94556863421184] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.063+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.063+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94556863422336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9c00 con 0x55ffbc137c00 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.063+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e9800 con 0x55ffbc4e9000 2024-04-27T04:34:48.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.064+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.064+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94556863422336] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.064+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94556863423488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563800 con 0x55ffbc4e9000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.064+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562800 con 0x55ffbc562000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.065+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94556863423488] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.065+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.065+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94556863424640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563000 con 0x55ffbc562000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.065+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562c00 con 0x55ffbc137c00 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.066+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94556863424640] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.066+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.066+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94556863425792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563400 con 0x55ffbc137c00 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.066+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562400 con 0x55ffbc4e9400 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.068+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.068+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94556863425792] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.068+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94556863426944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e8400 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.068+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e000 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.069+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.069+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94556863428096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e400 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.069+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e800 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.069+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94556863426944] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94556863428096] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94556863429248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56ec00 con 0x55ffbc4e9400 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=0 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=1 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=2 2024-04-27T04:34:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=3 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=4 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=5 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.070+0000 7f9a1b48da80 20 add_watcher() i=6 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.071+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94556863429248] v31'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.071+0000 7f9a1b48da80 20 add_watcher() i=7 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.071+0000 7f9a1b48da80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.071+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56f000 con 0x55ffbc137c00 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.071+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.071+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94556864839680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56f400 con 0x55ffbc137c00 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.072+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94556864839680] v31'134 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.072+0000 7f9a00405700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.072+0000 7f9a1b48da80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.072+0000 7f9a1b48da80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:48.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a1b48da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a1b48da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbc1d83c0 con 0x55ffbc56fc00 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a18c36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a18c36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47660/0 (socket says 172.21.15.191:47660) 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a18c36700 1 -- 172.21.15.191:0/1667477945 learned_addr learned my addr 172.21.15.191:0/1667477945 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a18c36700 1 -- 172.21.15.191:0/1667477945 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15eb60 con 0x55ffbc56fc00 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.073+0000 7f9a18c36700 1 --2- 172.21.15.191:0/1667477945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4c55c0 tx=0x55ffbc48df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae0137dcec134c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f99ffc04700 1 -- 172.21.15.191:0/1667477945 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc586000 con 0x55ffbc56fc00 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f99ffc04700 1 -- 172.21.15.191:0/1667477945 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc57e180 con 0x55ffbc56fc00 2024-04-27T04:34:48.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f99ffc04700 1 -- 172.21.15.191:0/1667477945 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc586160 con 0x55ffbc56fc00 2024-04-27T04:34:48.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 -- 172.21.15.191:0/1667477945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 msgr2=0x55ffbc420680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/1667477945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4c55c0 tx=0x55ffbc48df80 comp rx=0 tx=0).stop 2024-04-27T04:34:48.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 -- 172.21.15.191:0/1667477945 shutdown_connections 2024-04-27T04:34:48.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/1667477945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 -- 172.21.15.191:0/1667477945 >> 172.21.15.191:0/1667477945 conn(0x55ffbc56f800 msgr2=0x55ffbc224000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 -- 172.21.15.191:0/1667477945 shutdown_connections 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.074+0000 7f9a1b48da80 1 -- 172.21.15.191:0/1667477945 wait complete. 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a1b48da80 1 Processor -- start 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a1b48da80 1 -- start start 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a1b48da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a1b48da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbc1d8500 con 0x55ffbc56fc00 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a18c36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a18c36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47676/0 (socket says 172.21.15.191:47676) 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a18c36700 1 -- 172.21.15.191:0/754206917 learned_addr learned my addr 172.21.15.191:0/754206917 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a18c36700 1 -- 172.21.15.191:0/754206917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbc15ed00 con 0x55ffbc56fc00 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.075+0000 7f9a18c36700 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55ffbc575200 tx=0x55ffbc550240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f99fe401700 1 -- 172.21.15.191:0/754206917 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc5862c0 con 0x55ffbc56fc00 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f99fe401700 1 -- 172.21.15.191:0/754206917 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffbc57e300 con 0x55ffbc56fc00 2024-04-27T04:34:48.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ffbc15e340 con 0x55ffbc56fc00 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f99fe401700 1 -- 172.21.15.191:0/754206917 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ffbc586000 con 0x55ffbc56fc00 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbc15eb60 con 0x55ffbc56fc00 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f99fe401700 1 -- 172.21.15.191:0/754206917 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ffbced9b80 con 0x55ffbc56fc00 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f99fe401700 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc563c00 0x55ffbc421700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.076+0000 7f99fe401700 1 -- 172.21.15.191:0/754206917 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ffbc1b16c0 con 0x55ffbc56fc00 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.077+0000 7f9a19c38700 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc563c00 0x55ffbc421700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.077+0000 7f9a1b48da80 5 note: GC not initialized 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.077+0000 7f9a19c38700 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc563c00 0x55ffbc421700 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4c5f50 tx=0x55ffbc5512c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.077+0000 7f99fbbfc700 20 reqs_thread_entry: start 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.077+0000 7f9a1b48da80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f99f9bf8700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:48.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f99f93d6700 20 reqs_thread_entry: start 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e8400 con 0x55ffbc4e9400 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=385 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.078+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562c00 con 0x55ffbc562000 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc563400 con 0x55ffbc4e9400 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-04-27T04:34:48.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.079+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562400 con 0x55ffbc562000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.081+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 foo.buckets [call] v31'24 uv24 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.081+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 8.9 8:9670fbcc:root::.bucket.meta.testx%3amyfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [create,call version.inc in=38b,writefull 0~371 in=371b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e8400 con 0x55ffbc562000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [create,call,writefull 0~371,setxattr (203)] v31'25 uv25 ondisk = 0) v8 ==== 338+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a1b48da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x7 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a1b48da80 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a1b48da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a1b48da80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94556864841984 in=818b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e000 con 0x55ffbc4e9000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94556863423488 notify 133143986181 ret 0) v3 ==== 848+0+0 (crc 0 0 0) 0x55ffbc5029c0 con 0x55ffbc4e9000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a13c2c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94556863423488 notifier 4580 bl.length()=806 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(67 notify.2 [notify cookie 94556864841984 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a13c2c700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x7 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a13c2c700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a13c2c700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.082+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562800 con 0x55ffbc4e9000 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 94556864841984 notify 133143986181 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ffbc502b60 con 0x55ffbc4e9000 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a1b48da80 20 handle_bi_update(): bucket=testx:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_sources= new_sources= 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a1b48da80 20 handle_bi_update(): bucket=testx:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential sources added=[] removed=[] 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a1b48da80 20 handle_bi_update(): bucket=testx:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_dests= new_dests= 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a1b48da80 20 handle_bi_update(): bucket=testx:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential dests added=[] removed=[] 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.083+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e400 con 0x55ffbc137c00 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.084+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v31'1 uv1 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.084+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e800 con 0x55ffbc137c00 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v31'1 uv1 ondisk = 0) v8 ==== 281+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a1b48da80 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a1b48da80 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T04:34:48.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a1b48da80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 94556864841984 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56ec00 con 0x55ffbc137c00 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94556863425792 notify 133143986187 ret 0) v3 ==== 390+0+0 (crc 0 0 0) 0x55ffbc57a000 con 0x55ffbc137c00 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a13c2c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986187 cookie 94556863425792 notifier 4580 bl.length()=348 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(71 notify.4 [notify cookie 94556864841984 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a13c2c700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a13c2c700 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562c00 con 0x55ffbc137c00 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94556864841984 notify 133143986187 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ffbc5029c0 con 0x55ffbc137c00 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.085+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56f000 con 0x55ffbc562000 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(73 myfoo [call,delete] v31'26 uv26 ondisk = 0) v8 ==== 191+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a1b48da80 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a1b48da80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:34:48.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94556864841984 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56f400 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== watch-notify(notify (1) cookie 94556863428096 notify 133143986193 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x55ffbc5029c0 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a13c2c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986193 cookie 94556863428096 notifier 4580 bl.length()=149 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(74 notify.6 [notify cookie 94556864841984 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc4e8400 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== watch-notify(notify_complete (2) cookie 94556864841984 notify 133143986193 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ffbc57a000 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.087+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57c000 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.088+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(76 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call,delete] v31'4 uv4 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a1b48da80 10 removing default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 from cache 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a1b48da80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [notify cookie 94556864841984 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57c400 con 0x55ffbc562000 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== watch-notify(notify (1) cookie 94556863424640 notify 133143986192 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x55ffbcfd1860 con 0x55ffbc562000 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a13c2c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94556863424640 notifier 4580 bl.length()=206 2024-04-27T04:34:48.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(77 notify.3 [notify cookie 94556864841984 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a13c2c700 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc562400 con 0x55ffbc562000 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== watch-notify(notify_complete (2) cookie 94556864841984 notify 133143986192 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ffbcfd1a00 con 0x55ffbc562000 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.089+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(78 notify.3 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.090+0000 7f9a00405700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.090+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94556864839680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57c800 con 0x55ffbc137c00 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.091+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94556864839680] v31'135 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.091+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94556863421184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57cc00 con 0x55ffbc562000 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.092+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94556863421184] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.092+0000 7f9a1b48da80 20 remove_watcher() i=0 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.092+0000 7f9a1b48da80 2 removed watcher, disabling cache 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.092+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94556863422336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc56e000 con 0x55ffbc137c00 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.093+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94556863422336] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.093+0000 7f9a1b48da80 20 remove_watcher() i=1 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.093+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94556863423488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57d000 con 0x55ffbc4e9000 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.094+0000 7f9a19437700 1 -- 172.21.15.191:0/3041619860 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94556863423488] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d200 con 0x55ffbc4e9000 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.094+0000 7f9a1b48da80 20 remove_watcher() i=2 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.094+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94556863424640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57d400 con 0x55ffbc562000 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.095+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94556863424640] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55d440 con 0x55ffbc562000 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.095+0000 7f9a1b48da80 20 remove_watcher() i=3 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.095+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94556863425792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57d800 con 0x55ffbc137c00 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.096+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94556863425792] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc137c00 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.096+0000 7f9a1b48da80 20 remove_watcher() i=4 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.096+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94556863426944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57dc00 con 0x55ffbc4e9400 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.097+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94556863426944] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.097+0000 7f9a1b48da80 20 remove_watcher() i=5 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.097+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94556863428096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57c400 con 0x55ffbc4e9400 2024-04-27T04:34:48.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.098+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94556863428096] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.098+0000 7f9a1b48da80 20 remove_watcher() i=6 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.098+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94556863429248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ffbc57cc00 con 0x55ffbc4e9400 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a19c38700 1 -- 172.21.15.191:0/3041619860 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94556863429248] v31'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ffbc55cb40 con 0x55ffbc4e9400 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 20 remove_watcher() i=7 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ffbc4e9400 msgr2=0x55ffbc421180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ffbc4e9400 0x55ffbc421180 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ffbc137c00 msgr2=0x55ffbc41fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ffbc137c00 0x55ffbc41fb80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ffbc562000 msgr2=0x55ffbc420c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ffbc562000 0x55ffbc420c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ffbc4e9000 msgr2=0x55ffbc420100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ffbc4e9000 0x55ffbc420100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc4e8c00 msgr2=0x55ffbc41f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc4e8c00 0x55ffbc41f600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf06030 tx=0x55ffbc550000 comp rx=0 tx=0).stop 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 msgr2=0x55ffbc41eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/3041619860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc137800 0x55ffbc41eb00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4c69c0 tx=0x55ffbc48ce40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 reap_dead start 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a18c36700 1 -- 172.21.15.191:0/3041619860 reap_dead start 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 shutdown_connections 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 >> 172.21.15.191:0/3041619860 conn(0x55ffbc136c00 msgr2=0x55ffbc223800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 shutdown_connections 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.099+0000 7f9a1b48da80 1 -- 172.21.15.191:0/3041619860 wait complete. 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc563c00 msgr2=0x55ffbc421700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc563c00 0x55ffbc421700 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55ffbc4c5f50 tx=0x55ffbc5512c0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 msgr2=0x55ffbc420680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55ffbc575200 tx=0x55ffbc550240 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 shutdown_connections 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc563c00 0x55ffbc421700 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/754206917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc56fc00 0x55ffbc420680 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 >> 172.21.15.191:0/754206917 conn(0x55ffbc56f800 msgr2=0x55ffbc224000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 shutdown_connections 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/754206917 wait complete. 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc137400 msgr2=0x55ffbc41f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc137400 0x55ffbc41f080 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf07fb0 tx=0x55ffbc48cb40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 msgr2=0x55ffbc41e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55ffbcf07ec0 tx=0x55ffbc227980 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 shutdown_connections 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ffbc137400 0x55ffbc41f080 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 --2- 172.21.15.191:0/253519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbc136800 0x55ffbc41e580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 >> 172.21.15.191:0/253519168 conn(0x55ffbc136400 msgr2=0x55ffbc223000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 shutdown_connections 2024-04-27T04:34:48.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.100+0000 7f9a1b48da80 1 -- 172.21.15.191:0/253519168 wait complete. 2024-04-27T04:34:48.127 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-04-27T04:34:48.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-04-27T04:34:48.127 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-04-27T04:34:48.171 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.172 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f718030aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ba35384c00 0x55ba3566c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f718030aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ba356af500 con 0x55ba35384800 2024-04-27T04:34:48.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f718030aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ba35426a00 con 0x55ba35384c00 2024-04-27T04:34:48.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f717dab3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ba35384c00 0x55ba3566c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f717e2b4700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55ba35384800 0x55ba35470800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60496/0 (socket says 172.21.15.191:60496) 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f717e2b4700 1 -- 172.21.15.191:0/3033018438 learned_addr learned my addr 172.21.15.191:0/3033018438 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f717dab3700 1 -- 172.21.15.191:0/3033018438 >> v1:172.21.15.138:6789/0 conn(0x55ba35384800 legacy=0x55ba35470800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f717dab3700 1 -- 172.21.15.191:0/3033018438 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353add40 con 0x55ba35384c00 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.170+0000 7f717dab3700 1 --2- 172.21.15.191:0/3033018438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384c00 0x55ba3566c580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55ba35667890 tx=0x55ba35475920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e53db956db354a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f717d2b2700 1 -- 172.21.15.191:0/3033018438 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d4a0 con 0x55ba35384c00 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f717d2b2700 1 -- 172.21.15.191:0/3033018438 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba356af800 con 0x55ba35384c00 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f717d2b2700 1 -- 172.21.15.191:0/3033018438 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d600 con 0x55ba35384c00 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 -- 172.21.15.191:0/3033018438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384c00 msgr2=0x55ba3566c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 --2- 172.21.15.191:0/3033018438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384c00 0x55ba3566c580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55ba35667890 tx=0x55ba35475920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 -- 172.21.15.191:0/3033018438 shutdown_connections 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 --2- 172.21.15.191:0/3033018438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384c00 0x55ba3566c580 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 -- 172.21.15.191:0/3033018438 >> 172.21.15.191:0/3033018438 conn(0x55ba35384400 msgr2=0x55ba35470000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 -- 172.21.15.191:0/3033018438 shutdown_connections 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.171+0000 7f718030aa80 1 -- 172.21.15.191:0/3033018438 wait complete. 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f718030aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f718030aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba35426500 con 0x55ba35384800 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f717e2b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f717e2b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47684/0 (socket says 172.21.15.191:47684) 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f717e2b4700 1 -- 172.21.15.191:0/1721531224 learned_addr learned my addr 172.21.15.191:0/1721531224 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f717e2b4700 1 -- 172.21.15.191:0/1721531224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353adba0 con 0x55ba35384800 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.172+0000 7f717e2b4700 1 --2- 172.21.15.191:0/1721531224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55ba3614b800 tx=0x55ba35475b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36da8fe9b306e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f717cab1700 1 -- 172.21.15.191:0/1721531224 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d1e0 con 0x55ba35384800 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f717cab1700 1 -- 172.21.15.191:0/1721531224 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba356afb00 con 0x55ba35384800 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f717cab1700 1 -- 172.21.15.191:0/1721531224 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d4a0 con 0x55ba35384800 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 -- 172.21.15.191:0/1721531224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 msgr2=0x55ba3566c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 --2- 172.21.15.191:0/1721531224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55ba3614b800 tx=0x55ba35475b60 comp rx=0 tx=0).stop 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 -- 172.21.15.191:0/1721531224 shutdown_connections 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 --2- 172.21.15.191:0/1721531224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 -- 172.21.15.191:0/1721531224 >> 172.21.15.191:0/1721531224 conn(0x55ba35384400 msgr2=0x55ba35471000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 -- 172.21.15.191:0/1721531224 shutdown_connections 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 -- 172.21.15.191:0/1721531224 wait complete. 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.173+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f718030aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f718030aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba354263c0 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717e2b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717e2b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47692/0 (socket says 172.21.15.191:47692) 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717e2b4700 1 -- 172.21.15.191:0/3534109522 learned_addr learned my addr 172.21.15.191:0/3534109522 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717e2b4700 1 -- 172.21.15.191:0/3534109522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353ada00 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717e2b4700 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55ba3614ba10 tx=0x55ba35475a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717b2ae700 1 -- 172.21.15.191:0/3534109522 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d600 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717b2ae700 1 -- 172.21.15.191:0/3534109522 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba35425c80 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba353ad860 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f717b2ae700 1 -- 172.21.15.191:0/3534109522 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d1e0 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.174+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba353ad6c0 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717b2ae700 1 -- 172.21.15.191:0/3534109522 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ba36126b00 con 0x55ba35384800 2024-04-27T04:34:48.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f718030aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f718030aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba35426000 con 0x55ba35385400 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717b2ae700 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714000 0x55ba3566d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717b2ae700 1 -- 172.21.15.191:0/3534109522 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ba353ff180 con 0x55ba35384800 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717e2b4700 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714000 0x55ba3566d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717eab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717eab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47702/0 (socket says 172.21.15.191:47702) 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717eab5700 1 -- 172.21.15.191:0/1571284432 learned_addr learned my addr 172.21.15.191:0/1571284432 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717eab5700 1 -- 172.21.15.191:0/1571284432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353ac000 con 0x55ba35385400 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.175+0000 7f717e2b4700 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714000 0x55ba3566d080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55ba36202f90 tx=0x55ba356d0240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f717eab5700 1 --2- 172.21.15.191:0/1571284432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55ba36202300 tx=0x55ba356d10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7458d8fc98b9ae13 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f7178aa9700 1 -- 172.21.15.191:0/1571284432 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d600 con 0x55ba35385400 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f7178aa9700 1 -- 172.21.15.191:0/1571284432 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba356ae300 con 0x55ba35385400 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f7178aa9700 1 -- 172.21.15.191:0/1571284432 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d1e0 con 0x55ba35385400 2024-04-27T04:34:48.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 -- 172.21.15.191:0/1571284432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 msgr2=0x55ba3566cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 --2- 172.21.15.191:0/1571284432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55ba36202300 tx=0x55ba356d10e0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 -- 172.21.15.191:0/1571284432 shutdown_connections 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 --2- 172.21.15.191:0/1571284432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 -- 172.21.15.191:0/1571284432 >> 172.21.15.191:0/1571284432 conn(0x55ba35384c00 msgr2=0x55ba35471800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 -- 172.21.15.191:0/1571284432 shutdown_connections 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 -- 172.21.15.191:0/1571284432 wait complete. 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.176+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f718030aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f718030aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba353c0640 con 0x55ba35385400 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f717eab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f717eab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47712/0 (socket says 172.21.15.191:47712) 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 learned_addr learned my addr 172.21.15.191:0/1981966929 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353acea0 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f717eab5700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55ba362028d0 tx=0x55ba356d0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f71772a6700 1 -- 172.21.15.191:0/1981966929 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543da20 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f71772a6700 1 -- 172.21.15.191:0/1981966929 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba35425c80 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba353ac820 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f71772a6700 1 -- 172.21.15.191:0/1981966929 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d600 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.177+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba353ac9c0 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.178+0000 7f71772a6700 1 -- 172.21.15.191:0/1981966929 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ba36126dc0 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.178+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.178+0000 7f71772a6700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714c00 0x55ba3566d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.178+0000 7f71772a6700 1 -- 172.21.15.191:0/1981966929 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ba353ff340 con 0x55ba35385400 2024-04-27T04:34:48.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f717dab3700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714c00 0x55ba3566d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ba35385800 0x55ba3566db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35385c00 con 0x55ba35385800 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f717eab5700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ba35385800 0x55ba3566db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f717dab3700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714c00 0x55ba3566d600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55ba35667890 tx=0x55ba35475800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f717eab5700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ba35385800 0x55ba3566db80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f718030aa80 20 realm 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715000 con 0x55ba35385800 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.179+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715400 con 0x55ba35385800 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715800 con 0x55ba35385800 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ba35715c00 0x55ba3566e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715000 con 0x55ba35715c00 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f717e2b4700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ba35715c00 0x55ba3566e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.180+0000 7f717e2b4700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ba35715c00 0x55ba3566e100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f718030aa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f718030aa80 20 searching for the correct realm 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715400 con 0x55ba35715c00 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f717e2b4700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ba35715400 0x55ba3566ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715000 con 0x55ba35715400 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.181+0000 7f717dab3700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ba35715400 0x55ba3566ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717dab3700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ba35715400 0x55ba3566ec00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717dab3700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ba35715000 0x55ba3566f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717eab5700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ba35715000 0x55ba3566f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717eab5700 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ba35715000 0x55ba3566f180 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.182+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715400 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.183+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.184+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715400 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.185+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.186+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715400 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.187+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715400 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715c00 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.188+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715400 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35385c00 con 0x55ba35385800 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da400 con 0x55ba35385800 2024-04-27T04:34:48.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.189+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da800 con 0x55ba35715c00 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 20 zone default found 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 4 Realm: () 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 20 zonegroup default 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357dac00 con 0x55ba35715000 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.190+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db000 con 0x55ba35715c00 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db400 con 0x55ba35715c00 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 20 started sync module instance, tier type = 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.191+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db800 con 0x55ba35715400 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.192+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.192+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94258248170240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da800 con 0x55ba35715400 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.192+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db000 con 0x55ba35385800 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.193+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94258248170240] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.193+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.193+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94258248171392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db400 con 0x55ba35385800 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.193+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da400 con 0x55ba35715c00 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.194+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.194+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94258248172544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357dac00 con 0x55ba35715c00 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.194+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35385c00 con 0x55ba35715400 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.194+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94258248171392] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.195+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.195+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94258248172544] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.195+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94258248173696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35715800 con 0x55ba35715400 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.195+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35385800 2024-04-27T04:34:48.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.196+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.196+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94258248174848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db800 con 0x55ba35385800 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.196+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da800 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.196+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94258248173696] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.197+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.197+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94258248176000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357dbc00 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.197+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db000 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.197+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94258248174848] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.198+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94258248176000] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.198+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.198+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94258248177152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db400 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.198+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da000 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94258248177152] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94258248178304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db800 con 0x55ba35715000 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=0 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=1 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=2 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=3 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=4 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=5 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.199+0000 7f718030aa80 20 add_watcher() i=6 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.200+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94258248178304] v31'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.200+0000 7f718030aa80 20 add_watcher() i=7 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.200+0000 7f718030aa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.200+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da800 con 0x55ba35385800 2024-04-27T04:34:48.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.200+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.200+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94258249760768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357dbc00 con 0x55ba35385800 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.201+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94258249760768] v31'136 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.201+0000 7f7165282700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.201+0000 7f718030aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.201+0000 7f718030aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f718030aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f718030aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba353c0780 con 0x55ba357db400 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f717dab3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f717dab3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47718/0 (socket says 172.21.15.191:47718) 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f717dab3700 1 -- 172.21.15.191:0/3554701778 learned_addr learned my addr 172.21.15.191:0/3554701778 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f717dab3700 1 -- 172.21.15.191:0/3554701778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353acb60 con 0x55ba357db400 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f717dab3700 1 --2- 172.21.15.191:0/3554701778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55ba357f0060 tx=0x55ba35475980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e045394443e647 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.202+0000 7f7164a81700 1 -- 172.21.15.191:0/3554701778 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba3543d760 con 0x55ba357db400 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f7164a81700 1 -- 172.21.15.191:0/3554701778 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba357f8000 con 0x55ba357db400 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f7164a81700 1 -- 172.21.15.191:0/3554701778 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba357fa000 con 0x55ba357db400 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/3554701778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 msgr2=0x55ba3566e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 --2- 172.21.15.191:0/3554701778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55ba357f0060 tx=0x55ba35475980 comp rx=0 tx=0).stop 2024-04-27T04:34:48.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/3554701778 shutdown_connections 2024-04-27T04:34:48.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 --2- 172.21.15.191:0/3554701778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/3554701778 >> 172.21.15.191:0/3554701778 conn(0x55ba357db000 msgr2=0x55ba35472000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/3554701778 shutdown_connections 2024-04-27T04:34:48.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/3554701778 wait complete. 2024-04-27T04:34:48.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 Processor -- start 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- start start 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba35426000 con 0x55ba357db400 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f717dab3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f717dab3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47720/0 (socket says 172.21.15.191:47720) 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f717dab3700 1 -- 172.21.15.191:0/1491369338 learned_addr learned my addr 172.21.15.191:0/1491369338 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f717dab3700 1 -- 172.21.15.191:0/1491369338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba353acd00 con 0x55ba357db400 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f717dab3700 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55ba357f0270 tx=0x55ba354758c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f716327e700 1 -- 172.21.15.191:0/1491369338 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba357fa160 con 0x55ba357db400 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f716327e700 1 -- 172.21.15.191:0/1491369338 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba357f8180 con 0x55ba357db400 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f716327e700 1 -- 172.21.15.191:0/1491369338 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba357fa2c0 con 0x55ba357db400 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba353ac340 con 0x55ba357db400 2024-04-27T04:34:48.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.204+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba353add40 con 0x55ba357db400 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.205+0000 7f716327e700 1 -- 172.21.15.191:0/1491369338 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ba36127b80 con 0x55ba357db400 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.205+0000 7f716327e700 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35876000 0x55ba3566f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.205+0000 7f716327e700 1 -- 172.21.15.191:0/1491369338 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ba353ff500 con 0x55ba357db400 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.205+0000 7f717eab5700 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35876000 0x55ba3566f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.205+0000 7f718030aa80 5 note: GC not initialized 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.206+0000 7f717eab5700 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35876000 0x55ba3566f700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ba3577dce0 tx=0x55ba35805200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.206+0000 7f7160a79700 20 reqs_thread_entry: start 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.206+0000 7f718030aa80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.206+0000 7f718030aa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.206+0000 7f715ea75700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.206+0000 7f715e253700 20 reqs_thread_entry: start 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.207+0000 7f718030aa80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.207+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357db800 con 0x55ba35385800 2024-04-27T04:34:48.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.207+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 281+0+332 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.207+0000 7f718030aa80 20 rados_obj.operate() r=0 bl.length=193 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.208+0000 7f7165282700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.208+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94258249760768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba357da800 con 0x55ba35385800 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.209+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94258249760768] v31'137 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.209+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94258248170240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35876400 con 0x55ba35715400 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.210+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94258248170240] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.210+0000 7f718030aa80 20 remove_watcher() i=0 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.210+0000 7f718030aa80 2 removed watcher, disabling cache 2024-04-27T04:34:48.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.210+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94258248171392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35876800 con 0x55ba35385800 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.211+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94258248171392] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.211+0000 7f718030aa80 20 remove_watcher() i=1 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.211+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94258248172544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35876c00 con 0x55ba35715c00 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.212+0000 7f717e2b4700 1 -- 172.21.15.191:0/1981966929 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94258248172544] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5200 con 0x55ba35715c00 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.212+0000 7f718030aa80 20 remove_watcher() i=2 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.212+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94258248173696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35877000 con 0x55ba35715400 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.213+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94258248173696] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c5440 con 0x55ba35715400 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.213+0000 7f718030aa80 20 remove_watcher() i=3 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.213+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94258248174848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35877400 con 0x55ba35385800 2024-04-27T04:34:48.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.214+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94258248174848] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35385800 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.214+0000 7f718030aa80 20 remove_watcher() i=4 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.214+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94258248176000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35877800 con 0x55ba35715000 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.215+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94258248176000] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.215+0000 7f718030aa80 20 remove_watcher() i=5 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.215+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94258248177152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35877c00 con 0x55ba35715000 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.216+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94258248177152] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.216+0000 7f718030aa80 20 remove_watcher() i=6 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.216+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94258248178304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ba35876800 con 0x55ba35715000 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f717eab5700 1 -- 172.21.15.191:0/1981966929 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94258248178304] v31'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba357c4b40 con 0x55ba35715000 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 20 remove_watcher() i=7 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ba35715000 msgr2=0x55ba3566f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ba35715000 0x55ba3566f180 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ba35385800 msgr2=0x55ba3566db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ba35385800 0x55ba3566db80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ba35715400 msgr2=0x55ba3566ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ba35715400 0x55ba3566ec00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ba35715c00 msgr2=0x55ba3566e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ba35715c00 0x55ba3566e100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714c00 msgr2=0x55ba3566d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714c00 0x55ba3566d600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55ba35667890 tx=0x55ba35475800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 msgr2=0x55ba3566cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1981966929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35385400 0x55ba3566cb00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55ba362028d0 tx=0x55ba356d0cc0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 reap_dead start 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f717dab3700 1 -- 172.21.15.191:0/1981966929 reap_dead start 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 shutdown_connections 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 >> 172.21.15.191:0/1981966929 conn(0x55ba35384c00 msgr2=0x55ba35471800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 shutdown_connections 2024-04-27T04:34:48.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1981966929 wait complete. 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35876000 msgr2=0x55ba3566f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35876000 0x55ba3566f700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ba3577dce0 tx=0x55ba35805200 comp rx=0 tx=0).stop 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 msgr2=0x55ba3566e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55ba357f0270 tx=0x55ba354758c0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 shutdown_connections 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35876000 0x55ba3566f700 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/1491369338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba357db400 0x55ba3566e680 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 >> 172.21.15.191:0/1491369338 conn(0x55ba357db000 msgr2=0x55ba35472000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 shutdown_connections 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/1491369338 wait complete. 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714000 msgr2=0x55ba3566d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714000 0x55ba3566d080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55ba36202f90 tx=0x55ba356d0240 comp rx=0 tx=0).stop 2024-04-27T04:34:48.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 msgr2=0x55ba3566c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55ba3614ba10 tx=0x55ba35475a40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 shutdown_connections 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ba35714000 0x55ba3566d080 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 --2- 172.21.15.191:0/3534109522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba35384800 0x55ba3566c580 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 >> 172.21.15.191:0/3534109522 conn(0x55ba35384400 msgr2=0x55ba35471000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 shutdown_connections 2024-04-27T04:34:48.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.218+0000 7f718030aa80 1 -- 172.21.15.191:0/3534109522 wait complete. 2024-04-27T04:34:48.315 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:48.315 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "bucket:testx/myfoo", 2024-04-27T04:34:48.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": { 2024-04-27T04:34:48.315 INFO:teuthology.orchestra.run.smithi191.stdout: "tag": "Jd0HWijDTPxS2t3pGtRet6/o", 2024-04-27T04:34:48.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": 2 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "mtime": "2024-04-27T04:34:48.086103Z", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "data": { 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": { 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "myfoo", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "marker": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "tenant": "testx", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "explicit_placement": { 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "", 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "" 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:48.316 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:48.317 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "testx$tenanteduser", 2024-04-27T04:34:48.317 INFO:teuthology.orchestra.run.smithi191.stdout: "creation_time": "2024-04-27T04:34:45.731629Z", 2024-04-27T04:34:48.317 INFO:teuthology.orchestra.run.smithi191.stdout: "linked": true, 2024-04-27T04:34:48.317 INFO:teuthology.orchestra.run.smithi191.stdout: "has_bucket_info": false 2024-04-27T04:34:48.317 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:48.317 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:48.317 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'Jd0HWijDTPxS2t3pGtRet6/o', 'ver': 2}, 'mtime': '2024-04-27T04:34:48.086103Z', 'data': {'bucket': {'name': 'myfoo', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'bucket_id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-04-27T04:34:45.731629Z', 'linked': True, 'has_bucket_info': False}} 2024-04-27T04:34:48.317 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-27T04:34:48.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-27T04:34:48.318 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-04-27T04:34:48.364 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.365 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.363+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6d871a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6d871a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c77126f500 con 0x55c770f44c00 2024-04-27T04:34:48.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6d871a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c770fe6a00 con 0x55c770f44800 2024-04-27T04:34:48.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6b81b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6b01a700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55c770f44c00 0x55c771030800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60510/0 (socket says 172.21.15.191:60510) 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6b01a700 1 -- 172.21.15.191:0/1992953965 learned_addr learned my addr 172.21.15.191:0/1992953965 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6b81b700 1 -- 172.21.15.191:0/1992953965 >> v1:172.21.15.138:6789/0 conn(0x55c770f44c00 legacy=0x55c771030800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6b81b700 1 -- 172.21.15.191:0/1992953965 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6dd40 con 0x55c770f44800 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6a819700 1 -- 172.21.15.191:0/1992953965 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852419366 0 0) 0x55c771d12000 con 0x55c770f44c00 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.364+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/1992953965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55c7712279e0 tx=0x55c771035920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d81c062ed0e08c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6a819700 1 -- 172.21.15.191:0/1992953965 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd4a0 con 0x55c770f44800 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6a819700 1 -- 172.21.15.191:0/1992953965 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c77126f800 con 0x55c770f44800 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6a819700 1 -- 172.21.15.191:0/1992953965 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd600 con 0x55c770f44800 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1992953965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 msgr2=0x55c77122c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/1992953965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55c7712279e0 tx=0x55c771035920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1992953965 shutdown_connections 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/1992953965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1992953965 >> 172.21.15.191:0/1992953965 conn(0x55c770f44400 msgr2=0x55c771030000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1992953965 shutdown_connections 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.365+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1992953965 wait complete. 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6d871a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6d871a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c770fe6500 con 0x55c770f44800 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6b81b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6b81b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47740/0 (socket says 172.21.15.191:47740) 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3338170388 learned_addr learned my addr 172.21.15.191:0/3338170388 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3338170388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6dba0 con 0x55c770f44800 2024-04-27T04:34:48.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/3338170388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55c771d15800 tx=0x55c771035800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661514cdcbbedcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6a018700 1 -- 172.21.15.191:0/3338170388 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd760 con 0x55c770f44800 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6a018700 1 -- 172.21.15.191:0/3338170388 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c77126fb00 con 0x55c770f44800 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.366+0000 7fcb6a018700 1 -- 172.21.15.191:0/3338170388 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd1e0 con 0x55c770f44800 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3338170388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 msgr2=0x55c77122c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3338170388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55c771d15800 tx=0x55c771035800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3338170388 shutdown_connections 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3338170388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3338170388 >> 172.21.15.191:0/3338170388 conn(0x55c770f44400 msgr2=0x55c771031000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3338170388 shutdown_connections 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3338170388 wait complete. 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.367+0000 7fcb6d871a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c770fe63c0 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6b81b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6b81b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47750/0 (socket says 172.21.15.191:47750) 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6b81b700 1 -- 172.21.15.191:0/2218408919 learned_addr learned my addr 172.21.15.191:0/2218408919 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6b81b700 1 -- 172.21.15.191:0/2218408919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6da00 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55c771d15ec0 tx=0x55c771035920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb68815700 1 -- 172.21.15.191:0/2218408919 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd4a0 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb68815700 1 -- 172.21.15.191:0/2218408919 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c77126fc80 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c770f6d860 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb68815700 1 -- 172.21.15.191:0/2218408919 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd600 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.368+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c770f6d6c0 con 0x55c770f44800 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.391 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb68815700 1 -- 172.21.15.191:0/2218408919 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c771ce6b00 con 0x55c770f44800 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6d871a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6d871a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c770f80640 con 0x55c770f45400 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb68815700 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4000 0x55c77122d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb68815700 1 -- 172.21.15.191:0/2218408919 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c770fbf180 con 0x55c770f44800 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4000 0x55c77122d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6c01c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6c01c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47766/0 (socket says 172.21.15.191:47766) 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6c01c700 1 -- 172.21.15.191:0/1432909619 learned_addr learned my addr 172.21.15.191:0/1432909619 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6c01c700 1 -- 172.21.15.191:0/1432909619 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6c000 con 0x55c770f45400 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.369+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4000 0x55c77122d080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c771dbf050 tx=0x55c771298b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/1432909619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55c771dbe3f0 tx=0x55c771298e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e5fb011befdb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb66010700 1 -- 172.21.15.191:0/1432909619 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd4a0 con 0x55c770f45400 2024-04-27T04:34:48.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb66010700 1 -- 172.21.15.191:0/1432909619 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c770fe5c80 con 0x55c770f45400 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb66010700 1 -- 172.21.15.191:0/1432909619 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd600 con 0x55c770f45400 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1432909619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 msgr2=0x55c77122cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/1432909619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55c771dbe3f0 tx=0x55c771298e40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1432909619 shutdown_connections 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/1432909619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1432909619 >> 172.21.15.191:0/1432909619 conn(0x55c770f44c00 msgr2=0x55c771031800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1432909619 shutdown_connections 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.370+0000 7fcb6d871a80 1 -- 172.21.15.191:0/1432909619 wait complete. 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6d871a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6d871a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c770f808c0 con 0x55c770f45400 2024-04-27T04:34:48.393 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6c01c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6c01c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47778/0 (socket says 172.21.15.191:47778) 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 learned_addr learned my addr 172.21.15.191:0/3247248161 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6cea0 con 0x55c770f45400 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55c771dbe240 tx=0x55c771298a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6480d700 1 -- 172.21.15.191:0/3247248161 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd760 con 0x55c770f45400 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6480d700 1 -- 172.21.15.191:0/3247248161 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c770fe5b00 con 0x55c770f45400 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c770f6c820 con 0x55c770f45400 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.371+0000 7fcb6480d700 1 -- 172.21.15.191:0/3247248161 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c770ffd4a0 con 0x55c770f45400 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.372+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c770f6c9c0 con 0x55c770f45400 2024-04-27T04:34:48.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.372+0000 7fcb6480d700 1 -- 172.21.15.191:0/3247248161 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c771ce6dc0 con 0x55c770f45400 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.372+0000 7fcb6480d700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4c00 0x55c77122d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.372+0000 7fcb6480d700 1 -- 172.21.15.191:0/3247248161 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c770fbf340 con 0x55c770f45400 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.372+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4c00 0x55c77122d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4c00 0x55c77122d600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55c771d14030 tx=0x55c77137c0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c770f45800 0x55c77122db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c770f45c00 con 0x55c770f45800 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c770f45800 0x55c77122db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c770f45800 0x55c77122db80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 20 realm 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.373+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5000 con 0x55c770f45800 2024-04-27T04:34:48.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5400 con 0x55c770f45800 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5800 con 0x55c770f45800 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c7712d5c00 0x55c77122e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5000 con 0x55c7712d5c00 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.374+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c7712d5c00 0x55c77122e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c7712d5c00 0x55c77122e100 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6d871a80 20 searching for the correct realm 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5400 con 0x55c7712d5c00 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6b81b700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c7712d5400 0x55c77122ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5000 con 0x55c7712d5400 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.375+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c7712d5400 0x55c77122ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c7712d5400 0x55c77122ec00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c7712d5000 0x55c77122f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c7712d5000 0x55c77122f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c7712d5000 0x55c77122f180 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.376+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.377+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5400 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.378+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5400 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.379+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.379+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.379+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.379+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.379+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.379+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.380+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5400 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.381+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5400 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5c00 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.382+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5400 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:48.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c770f45c00 con 0x55c770f45800 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.383+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396400 con 0x55c770f45800 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396800 con 0x55c7712d5c00 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 zone default found 2024-04-27T04:34:48.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 4 Realm: () 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 zonegroup default 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396c00 con 0x55c7712d5000 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397000 con 0x55c7712d5c00 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.384+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397400 con 0x55c7712d5c00 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.385+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.385+0000 7fcb6d871a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.385+0000 7fcb6d871a80 20 started sync module instance, tier type = 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.385+0000 7fcb6d871a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.385+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397800 con 0x55c7712d5400 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.386+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.386+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94315084921600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396800 con 0x55c7712d5400 2024-04-27T04:34:48.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.386+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397000 con 0x55c770f45800 2024-04-27T04:34:48.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.387+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94315084921600] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.387+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.387+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94315084922752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397400 con 0x55c770f45800 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.387+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396400 con 0x55c7712d5c00 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.388+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.388+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94315084923904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396c00 con 0x55c7712d5c00 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.388+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c770f45c00 con 0x55c7712d5400 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.388+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94315084922752] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.389+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.389+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94315084923904] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.389+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94315084925056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7712d5800 con 0x55c7712d5400 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.389+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c770f45800 2024-04-27T04:34:48.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.390+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94315084925056] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.390+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.390+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94315084926208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397800 con 0x55c770f45800 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.390+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396800 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.391+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94315084926208] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.391+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.391+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94315084927360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397c00 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.391+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397000 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.392+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.392+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94315084928512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397400 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.392+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396000 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.392+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94315084927360] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94315084928512] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94315084929664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397800 con 0x55c7712d5000 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=0 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=1 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=2 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=3 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=4 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=5 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=6 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94315084929664] v31'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 20 add_watcher() i=7 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.394+0000 7fcb6d871a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.395+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396800 con 0x55c770f45800 2024-04-27T04:34:48.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.395+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.395+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94315086487552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397c00 con 0x55c770f45800 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94315086487552] v31'138 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb527e9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6d871a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6d871a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6d871a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.396+0000 7fcb6d871a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c770fe63c0 con 0x55c771397400 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb6b01a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb6b01a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47780/0 (socket says 172.21.15.191:47780) 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3717537888 learned_addr learned my addr 172.21.15.191:0/3717537888 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3717537888 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6cb60 con 0x55c771397400 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/3717537888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55c77132b800 tx=0x55c771299f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ce2106a67e894f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb51fe8700 1 -- 172.21.15.191:0/3717537888 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c7713b6000 con 0x55c771397400 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb51fe8700 1 -- 172.21.15.191:0/3717537888 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7713b8000 con 0x55c771397400 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.397+0000 7fcb51fe8700 1 -- 172.21.15.191:0/3717537888 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c7713b6160 con 0x55c771397400 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3717537888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 msgr2=0x55c77122e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3717537888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55c77132b800 tx=0x55c771299f80 comp rx=0 tx=0).stop 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3717537888 shutdown_connections 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3717537888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3717537888 >> 172.21.15.191:0/3717537888 conn(0x55c771397000 msgr2=0x55c771032000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3717537888 shutdown_connections 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3717537888 wait complete. 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 Processor -- start 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- start start 2024-04-27T04:34:48.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.398+0000 7fcb6d871a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c770fe6500 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6b01a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6b01a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47784/0 (socket says 172.21.15.191:47784) 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6b01a700 1 -- 172.21.15.191:0/2888893987 learned_addr learned my addr 172.21.15.191:0/2888893987 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6b01a700 1 -- 172.21.15.191:0/2888893987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c770f6cd00 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6b01a700 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55c7713a7200 tx=0x55c77137c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb507e5700 1 -- 172.21.15.191:0/2888893987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c7713b62c0 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb507e5700 1 -- 172.21.15.191:0/2888893987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7713b8180 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c770f6c340 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb507e5700 1 -- 172.21.15.191:0/2888893987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c7713b6000 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.399+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c770f6cb60 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.400+0000 7fcb507e5700 1 -- 172.21.15.191:0/2888893987 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c771ce7b80 con 0x55c771397400 2024-04-27T04:34:48.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.400+0000 7fcb507e5700 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c771438000 0x55c77122f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.400+0000 7fcb507e5700 1 -- 172.21.15.191:0/2888893987 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c770fbf500 con 0x55c771397400 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.400+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c771438000 0x55c77122f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.400+0000 7fcb6d871a80 5 note: GC not initialized 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.400+0000 7fcb6c01c700 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c771438000 0x55c77122f700 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55c77132b830 tx=0x55c77137c360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb4dfe0700 20 reqs_thread_entry: start 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb6d871a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb6d871a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb4bfdc700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb4b7ba700 20 reqs_thread_entry: start 2024-04-27T04:34:48.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb6d871a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.401+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397800 con 0x55c770f45800 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396800 con 0x55c770f45800 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+241 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=193 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.402+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:9670fbcc:root::.bucket.meta.testx%3amyfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771438400 con 0x55c7712d5400 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~371 out=371b,getxattrs out=298b] v0'0 uv25 ondisk = 0) v8 ==== 338+0+733 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 20 rados_obj.operate() r=0 bl.length=371 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T04:34:48.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.403+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771438800 con 0x55c770f45800 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.404+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v31'2 uv2 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.404+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [create,call version.inc in=38b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771438c00 con 0x55c7712d5000 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(66 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [create,call,writefull 0~352,setxattr (123)] v31'5 uv5 ondisk = 0) v8 ==== 332+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6d871a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x7 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6d871a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6d871a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6d871a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [notify cookie 94315086489856 in=713b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771439000 con 0x55c7712d5400 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== watch-notify(notify (1) cookie 94315084925056 notify 133143986193 ret 0) v3 ==== 743+0+0 (crc 0 0 0) 0x55c7713361a0 con 0x55c7712d5400 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.405+0000 7fcb66010700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986193 cookie 94315084925056 notifier 4596 bl.length()=701 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb66010700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x7 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 notify.3 [notify cookie 94315086489856 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb66010700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb66010700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:48.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb66010700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b2000 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.3 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== watch-notify(notify_complete (2) cookie 94315086489856 notify 133143986193 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c771336340 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6d871a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_sources= new_sources= 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6d871a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential sources added=[] removed=[] 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6d871a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_dests= new_dests= 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6d871a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential dests added=[] removed=[] 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.406+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771439400 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.407+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(69 foo.buckets [call] v31'27 uv27 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.407+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771439800 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v31'28 uv28 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6d871a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6d871a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:48.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6d871a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94315086489856 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771439c00 con 0x55c7712d5000 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== watch-notify(notify (1) cookie 94315084928512 notify 133143986194 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55c771337d40 con 0x55c7712d5000 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb66010700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986194 cookie 94315084928512 notifier 4596 bl.length()=314 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(71 notify.6 [notify cookie 94315086489856 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb66010700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb66010700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.408+0000 7fcb66010700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b2400 con 0x55c7712d5000 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.409+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.409+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== watch-notify(notify_complete (2) cookie 94315086489856 notify 133143986194 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c771337a00 con 0x55c7712d5000 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.409+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771438800 con 0x55c770f45800 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(73 testx/myfoo [call,delete] v31'2 uv2 ondisk = 0) v8 ==== 197+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6d871a80 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6d871a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94315086489856 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771438c00 con 0x55c770f45800 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== watch-notify(notify (1) cookie 94315084926208 notify 133143986188 ret 0) v3 ==== 197+0+0 (crc 0 0 0) 0x55c771337a00 con 0x55c770f45800 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb66010700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986188 cookie 94315084926208 notifier 4596 bl.length()=155 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(74 notify.4 [notify cookie 94315086489856 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb66010700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b2000 con 0x55c770f45800 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== watch-notify(notify_complete (2) cookie 94315086489856 notify 133143986188 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c771337ba0 con 0x55c770f45800 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.410+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 8.9 8:9670fbcc:root::.bucket.meta.testx%3amyfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771439c00 con 0x55c7712d5400 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.411+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call,delete] v31'29 uv29 ondisk = 0) v8 ==== 254+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.411+0000 7fcb6d871a80 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 from cache 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.411+0000 7fcb6d871a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.411+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [notify cookie 94315086489856 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771438400 con 0x55c7712d5c00 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.412+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94315084923904 notify 133143986182 ret 0) v3 ==== 254+0+0 (crc 0 0 0) 0x55c77128dba0 con 0x55c7712d5c00 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.412+0000 7fcb6580f700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986182 cookie 94315084923904 notifier 4596 bl.length()=212 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.412+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(77 notify.2 [notify cookie 94315086489856 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.412+0000 7fcb6580f700 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b2c00 con 0x55c7712d5c00 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.412+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== watch-notify(notify_complete (2) cookie 94315086489856 notify 133143986182 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c77128dd40 con 0x55c7712d5c00 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.412+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(78 notify.2 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.413+0000 7fcb527e9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.413+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94315086487552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771439000 con 0x55c770f45800 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.413+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94315086487552] v31'139 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.414+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94315084921600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c770f45c00 con 0x55c7712d5400 2024-04-27T04:34:48.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.414+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94315084921600] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.414+0000 7fcb6d871a80 20 remove_watcher() i=0 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.414+0000 7fcb6d871a80 2 removed watcher, disabling cache 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.414+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94315084922752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397c00 con 0x55c770f45800 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.416+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94315084922752] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.416+0000 7fcb6d871a80 20 remove_watcher() i=1 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.416+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94315084923904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771397800 con 0x55c7712d5c00 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.417+0000 7fcb6b81b700 1 -- 172.21.15.191:0/3247248161 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94315084923904] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f200 con 0x55c7712d5c00 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.417+0000 7fcb6d871a80 20 remove_watcher() i=2 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.417+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94315084925056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c771396800 con 0x55c7712d5400 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.418+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94315084925056] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138f440 con 0x55c7712d5400 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.418+0000 7fcb6d871a80 20 remove_watcher() i=3 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.418+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94315084926208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b2c00 con 0x55c770f45800 2024-04-27T04:34:48.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.419+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94315084926208] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c770f45800 2024-04-27T04:34:48.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.419+0000 7fcb6d871a80 20 remove_watcher() i=4 2024-04-27T04:34:48.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.419+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94315084927360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b3000 con 0x55c7712d5000 2024-04-27T04:34:48.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.420+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94315084927360] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.421+0000 7fcb6d871a80 20 remove_watcher() i=5 2024-04-27T04:34:48.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.421+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94315084928512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b3400 con 0x55c7712d5000 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.422+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94315084928512] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.422+0000 7fcb6d871a80 20 remove_watcher() i=6 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.422+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94315084929664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c7714b3800 con 0x55c7712d5000 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6c01c700 1 -- 172.21.15.191:0/3247248161 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94315084929664] v31'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c77138eb40 con 0x55c7712d5000 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 20 remove_watcher() i=7 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c7712d5000 msgr2=0x55c77122f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55c7712d5000 0x55c77122f180 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c770f45800 msgr2=0x55c77122db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c770f45800 0x55c77122db80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c7712d5400 msgr2=0x55c77122ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55c7712d5400 0x55c77122ec00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c7712d5c00 msgr2=0x55c77122e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c7712d5c00 0x55c77122e100 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4c00 msgr2=0x55c77122d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4c00 0x55c77122d600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55c771d14030 tx=0x55c77137c0c0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 msgr2=0x55c77122cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/3247248161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f45400 0x55c77122cb00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55c771dbe240 tx=0x55c771298a20 comp rx=0 tx=0).stop 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 reap_dead start 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6b01a700 1 -- 172.21.15.191:0/3247248161 reap_dead start 2024-04-27T04:34:48.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 shutdown_connections 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 >> 172.21.15.191:0/3247248161 conn(0x55c770f44c00 msgr2=0x55c771031800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 shutdown_connections 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/3247248161 wait complete. 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c771438000 msgr2=0x55c77122f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c771438000 0x55c77122f700 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55c77132b830 tx=0x55c77137c360 comp rx=0 tx=0).stop 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 msgr2=0x55c77122e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55c7713a7200 tx=0x55c77137c4e0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 shutdown_connections 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c771438000 0x55c77122f700 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2888893987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c771397400 0x55c77122e680 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.423+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 >> 172.21.15.191:0/2888893987 conn(0x55c771397000 msgr2=0x55c771032000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 shutdown_connections 2024-04-27T04:34:48.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2888893987 wait complete. 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4000 msgr2=0x55c77122d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4000 0x55c77122d080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c771dbf050 tx=0x55c771298b40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 msgr2=0x55c77122c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55c771d15ec0 tx=0x55c771035920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 shutdown_connections 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c7712d4000 0x55c77122d080 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 --2- 172.21.15.191:0/2218408919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c770f44800 0x55c77122c580 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 >> 172.21.15.191:0/2218408919 conn(0x55c770f44400 msgr2=0x55c771031000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 shutdown_connections 2024-04-27T04:34:48.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.424+0000 7fcb6d871a80 1 -- 172.21.15.191:0/2218408919 wait complete. 2024-04-27T04:34:48.445 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T04:34:48.446 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T04:34:48.446 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-04-27T04:34:48.493 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.494 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3d1614a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55aabee28c00 0x55aabf110580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3d1614a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55aabf153500 con 0x55aabee28800 2024-04-27T04:34:48.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3d1614a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55aabeecaa00 con 0x55aabee28c00 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3cedbd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55aabee28c00 0x55aabf110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3cedbd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55aabee28c00 0x55aabf110580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47788/0 (socket says 172.21.15.191:47788) 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3cedbd700 1 -- 172.21.15.191:0/4089592228 learned_addr learned my addr 172.21.15.191:0/4089592228 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3cf5be700 1 --1- 172.21.15.191:0/4089592228 >> v1:172.21.15.138:6789/0 conn(0x55aabee28800 0x55aabef14800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60518/0 (socket says 172.21.15.191:60518) 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.493+0000 7fe3cedbd700 1 -- 172.21.15.191:0/4089592228 >> v1:172.21.15.138:6789/0 conn(0x55aabee28800 legacy=0x55aabef14800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3cedbd700 1 -- 172.21.15.191:0/4089592228 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee51d40 con 0x55aabee28c00 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/4089592228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28c00 0x55aabf110580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55aabf10b890 tx=0x55aabef19920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84214f433d6486f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3ce5bc700 1 -- 172.21.15.191:0/4089592228 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee14a0 con 0x55aabee28c00 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3ce5bc700 1 -- 172.21.15.191:0/4089592228 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabf153800 con 0x55aabee28c00 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3ce5bc700 1 -- 172.21.15.191:0/4089592228 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee1600 con 0x55aabee28c00 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4089592228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28c00 msgr2=0x55aabf110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/4089592228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28c00 0x55aabf110580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55aabf10b890 tx=0x55aabef19920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4089592228 shutdown_connections 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/4089592228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28c00 0x55aabf110580 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4089592228 >> 172.21.15.191:0/4089592228 conn(0x55aabee28400 msgr2=0x55aabef14000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4089592228 shutdown_connections 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.494+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4089592228 wait complete. 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3d1614a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3d1614a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabeeca500 con 0x55aabee28800 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3cf5be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3cf5be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47790/0 (socket says 172.21.15.191:47790) 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.495+0000 7fe3cf5be700 1 -- 172.21.15.191:0/1610964836 learned_addr learned my addr 172.21.15.191:0/1610964836 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3cf5be700 1 -- 172.21.15.191:0/1610964836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee51ba0 con 0x55aabee28800 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/1610964836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55aabfbf3890 tx=0x55aabef19b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f6f31077c67c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3cddbb700 1 -- 172.21.15.191:0/1610964836 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee11e0 con 0x55aabee28800 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3cddbb700 1 -- 172.21.15.191:0/1610964836 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabf153b00 con 0x55aabee28800 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3cddbb700 1 -- 172.21.15.191:0/1610964836 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee14a0 con 0x55aabee28800 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 -- 172.21.15.191:0/1610964836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 msgr2=0x55aabf110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/1610964836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55aabfbf3890 tx=0x55aabef19b60 comp rx=0 tx=0).stop 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 -- 172.21.15.191:0/1610964836 shutdown_connections 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/1610964836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 -- 172.21.15.191:0/1610964836 >> 172.21.15.191:0/1610964836 conn(0x55aabee28400 msgr2=0x55aabef15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 -- 172.21.15.191:0/1610964836 shutdown_connections 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.496+0000 7fe3d1614a80 1 -- 172.21.15.191:0/1610964836 wait complete. 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3d1614a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3d1614a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabeeca3c0 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3cf5be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3cf5be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47802/0 (socket says 172.21.15.191:47802) 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3cf5be700 1 -- 172.21.15.191:0/799238345 learned_addr learned my addr 172.21.15.191:0/799238345 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.497+0000 7fe3cf5be700 1 -- 172.21.15.191:0/799238345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee51a00 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55aabfbf36b0 tx=0x55aabef19a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3cc5b8700 1 -- 172.21.15.191:0/799238345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee1600 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3cc5b8700 1 -- 172.21.15.191:0/799238345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabeec9c80 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aabee51860 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3cc5b8700 1 -- 172.21.15.191:0/799238345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee11e0 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aabee516c0 con 0x55aabee28800 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3cc5b8700 1 -- 172.21.15.191:0/799238345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55aabfbcab00 con 0x55aabee28800 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3d1614a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.498+0000 7fe3d1614a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabee64640 con 0x55aabee29400 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cc5b8700 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1ba000 0x55aabf111080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cc5b8700 1 -- 172.21.15.191:0/799238345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55aabeea3180 con 0x55aabee28800 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cfdbf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1ba000 0x55aabf111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cfdbf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47808/0 (socket says 172.21.15.191:47808) 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3496740981 learned_addr learned my addr 172.21.15.191:0/3496740981 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3496740981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee50000 con 0x55aabee29400 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1ba000 0x55aabf111080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55aabfca7020 tx=0x55aabf174240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3496740981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55aabfca63f0 tx=0x55aabf1750e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d48b61fcc37b700 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3c9db3700 1 -- 172.21.15.191:0/3496740981 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee1600 con 0x55aabee29400 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3c9db3700 1 -- 172.21.15.191:0/3496740981 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabf152300 con 0x55aabee29400 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.499+0000 7fe3c9db3700 1 -- 172.21.15.191:0/3496740981 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee11e0 con 0x55aabee29400 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3496740981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 msgr2=0x55aabf110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3496740981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55aabfca63f0 tx=0x55aabf1750e0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3496740981 shutdown_connections 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3496740981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3496740981 >> 172.21.15.191:0/3496740981 conn(0x55aabee28c00 msgr2=0x55aabef15800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3496740981 shutdown_connections 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3496740981 wait complete. 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3d1614a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabee648c0 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3cfdbf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3cfdbf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47810/0 (socket says 172.21.15.191:47810) 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.500+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 learned_addr learned my addr 172.21.15.191:0/3250898744 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee50ea0 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55aabfca6750 tx=0x55aabf174900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3c85b0700 1 -- 172.21.15.191:0/3250898744 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee1a20 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3c85b0700 1 -- 172.21.15.191:0/3250898744 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabeec9c80 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aabee50820 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3c85b0700 1 -- 172.21.15.191:0/3250898744 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee1600 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.501+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aabee509c0 con 0x55aabee29400 2024-04-27T04:34:48.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3c85b0700 1 -- 172.21.15.191:0/3250898744 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55aabfbcadc0 con 0x55aabee29400 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3c85b0700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1bac00 0x55aabf111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3c85b0700 1 -- 172.21.15.191:0/3250898744 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55aabeea3340 con 0x55aabee29400 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1bac00 0x55aabf111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1bac00 0x55aabf111600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55aabf10b890 tx=0x55aabef19800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabee29800 0x55aabf111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabee29c00 con 0x55aabee29800 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.502+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabee29800 0x55aabf111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabee29800 0x55aabf111b80 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 realm 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb000 con 0x55aabee29800 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb400 con 0x55aabee29800 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.503+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb800 con 0x55aabee29800 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf1bbc00 0x55aabf112100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb000 con 0x55aabf1bbc00 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf1bbc00 0x55aabf112100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf1bbc00 0x55aabf112100 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 20 searching for the correct realm 2024-04-27T04:34:48.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.504+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb400 con 0x55aabf1bbc00 2024-04-27T04:34:48.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf1bb400 0x55aabf112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb000 con 0x55aabf1bb400 2024-04-27T04:34:48.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf1bb400 0x55aabf112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf1bb400 0x55aabf112c00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf1bb000 0x55aabf113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.505+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf1bb000 0x55aabf113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.506+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf1bb000 0x55aabf113180 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.506+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.506+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.506+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.506+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.506+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb400 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.507+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb400 2024-04-27T04:34:48.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.508+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.509+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb400 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.510+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.511+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb400 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bbc00 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb400 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3d1614a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3d1614a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3d1614a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.512+0000 7fe3d1614a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabee29c00 con 0x55aabee29800 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e400 con 0x55aabee29800 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e800 con 0x55aabf1bbc00 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:48.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 zone default found 2024-04-27T04:34:48.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 4 Realm: () 2024-04-27T04:34:48.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:48.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:48.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:48.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 zonegroup default 2024-04-27T04:34:48.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.513+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabf1bb000 2024-04-27T04:34:48.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f000 con 0x55aabf1bbc00 2024-04-27T04:34:48.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f400 con 0x55aabf1bbc00 2024-04-27T04:34:48.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 started sync module instance, tier type = 2024-04-27T04:34:48.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:48.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.514+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf1bb400 2024-04-27T04:34:48.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.515+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.515+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94191838329600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e800 con 0x55aabf1bb400 2024-04-27T04:34:48.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.515+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f000 con 0x55aabee29800 2024-04-27T04:34:48.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.516+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94191838329600] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.516+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.516+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94191838330752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f400 con 0x55aabee29800 2024-04-27T04:34:48.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.516+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e400 con 0x55aabf1bbc00 2024-04-27T04:34:48.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.517+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94191838330752] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.518+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.518+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94191838331904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabf1bbc00 2024-04-27T04:34:48.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.518+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabee29c00 con 0x55aabf1bb400 2024-04-27T04:34:48.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.518+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94191838331904] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.519+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.519+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94191838333056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf1bb800 con 0x55aabf1bb400 2024-04-27T04:34:48.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.519+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabee29800 2024-04-27T04:34:48.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.520+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.520+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94191838334208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabee29800 2024-04-27T04:34:48.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.520+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e800 con 0x55aabf1bb000 2024-04-27T04:34:48.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.520+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94191838333056] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.521+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94191838334208] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.521+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.521+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94191838335360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27fc00 con 0x55aabf1bb000 2024-04-27T04:34:48.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.521+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f000 con 0x55aabf1bb000 2024-04-27T04:34:48.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.522+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.522+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94191838336512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f400 con 0x55aabf1bb000 2024-04-27T04:34:48.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.522+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e000 con 0x55aabf1bb000 2024-04-27T04:34:48.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.522+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94191838335360] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94191838336512] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94191838337664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf1bb000 2024-04-27T04:34:48.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=0 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=1 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=2 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=3 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=4 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=5 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=6 2024-04-27T04:34:48.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94191838337664] v31'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 20 add_watcher() i=7 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.523+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e800 con 0x55aabee29800 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.524+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.524+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94191839920128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27fc00 con 0x55aabee29800 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94191839920128] v31'140 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3b658c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3d1614a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3d1614a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3d1614a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.525+0000 7fe3d1614a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabeeca3c0 con 0x55aabf27f400 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3cedbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3cedbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47822/0 (socket says 172.21.15.191:47822) 2024-04-27T04:34:48.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3cedbd700 1 -- 172.21.15.191:0/4041642084 learned_addr learned my addr 172.21.15.191:0/4041642084 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3cedbd700 1 -- 172.21.15.191:0/4041642084 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee50b60 con 0x55aabf27f400 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/4041642084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55aabf211860 tx=0x55aabef19980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2b9a8052aacc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3b5d8b700 1 -- 172.21.15.191:0/4041642084 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabeee1760 con 0x55aabf27f400 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3b5d8b700 1 -- 172.21.15.191:0/4041642084 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabf29a000 con 0x55aabf27f400 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.526+0000 7fe3b5d8b700 1 -- 172.21.15.191:0/4041642084 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabf29c000 con 0x55aabf27f400 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4041642084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 msgr2=0x55aabf112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/4041642084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55aabf211860 tx=0x55aabef19980 comp rx=0 tx=0).stop 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4041642084 shutdown_connections 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/4041642084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4041642084 >> 172.21.15.191:0/4041642084 conn(0x55aabf27f000 msgr2=0x55aabef16000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4041642084 shutdown_connections 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 -- 172.21.15.191:0/4041642084 wait complete. 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 Processor -- start 2024-04-27T04:34:48.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.527+0000 7fe3d1614a80 1 -- start start 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3d1614a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3d1614a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabeeca500 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3cedbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3cedbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47828/0 (socket says 172.21.15.191:47828) 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 learned_addr learned my addr 172.21.15.191:0/3372351259 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabee50d00 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55aabf290630 tx=0x55aabef198c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3b4588700 1 -- 172.21.15.191:0/3372351259 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabf29c160 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3b4588700 1 -- 172.21.15.191:0/3372351259 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabf29a180 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aabee50340 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3b4588700 1 -- 172.21.15.191:0/3372351259 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aabf29c2c0 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.528+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aabee51d40 con 0x55aabf27f400 2024-04-27T04:34:48.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3b4588700 1 -- 172.21.15.191:0/3372351259 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55aabfbcbb80 con 0x55aabf27f400 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3b4588700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf316000 0x55aabf113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3b4588700 1 -- 172.21.15.191:0/3372351259 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55aabeea3500 con 0x55aabf27f400 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf316000 0x55aabf113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabf27e000 0x55aabf332000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf27e000 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.529+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabf27e000 0x55aabf332000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.530+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf316000 0x55aabf113700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55aabf2118c0 tx=0x55aabf29d320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.530+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabf27e000 0x55aabf332000 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.531+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'49 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf27e000 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.531+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf27e800 0x55aabf332580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.531+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf316400 con 0x55aabf27e800 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.531+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf27e800 0x55aabf332580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.531+0000 7fe3cedbd700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf27e800 0x55aabf332580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.532+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'22 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.532+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf316800 0x55aabf332b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.532+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf316c00 con 0x55aabf316800 2024-04-27T04:34:48.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.532+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf316800 0x55aabf332b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.532+0000 7fe3cfdbf700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf316800 0x55aabf332b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.533+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.533+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317000 con 0x55aabf27e000 2024-04-27T04:34:48.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.534+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'141 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf27e000 2024-04-27T04:34:48.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.534+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf317400 0x55aabf333080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.534+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317800 con 0x55aabf317400 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.534+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf317400 0x55aabf333080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.534+0000 7fe3cf5be700 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf317400 0x55aabf333080 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.535+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'47 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.535+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317c00 con 0x55aabf316800 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.536+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'38 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.536+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317000 con 0x55aabf27e000 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.538+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf27e000 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.538+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e400 con 0x55aabf27e800 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.539+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'33 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.539+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabf317400 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.540+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.540+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf316800 2024-04-27T04:34:48.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.541+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.541+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360000 con 0x55aabf317400 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.542+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'48 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.542+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360400 con 0x55aabf27e800 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.543+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.543+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360800 con 0x55aabf317400 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.544+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.544+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360c00 con 0x55aabf27e800 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.545+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'47 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.545+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361000 con 0x55aabf27e000 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.546+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'142 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf27e000 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.546+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361400 con 0x55aabf316800 2024-04-27T04:34:48.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.546+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'38 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.547+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361800 con 0x55aabf317400 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.547+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'49 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.547+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361c00 con 0x55aabf316800 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.548+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'28 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.548+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360000 con 0x55aabf27e000 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.549+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf27e000 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.549+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360800 con 0x55aabf27e000 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.550+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'143 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf27e000 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.550+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360400 con 0x55aabf316800 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.551+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'39 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.551+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabf317400 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.552+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'32 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.552+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e400 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.553+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.553+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317800 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.554+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'34 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.554+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317000 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.554+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.554+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf316400 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.555+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'39 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.555+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabee29c00 con 0x55aabf27e800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.556+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.556+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361400 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.557+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'28 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.557+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361c00 con 0x55aabf27e800 2024-04-27T04:34:48.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.558+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'34 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.558+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360400 con 0x55aabf316800 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.559+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3372351259 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'40 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf316800 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.559+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27fc00 con 0x55aabf27e800 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.560+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3372351259 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'48 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf27e800 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.560+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf317400 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.561+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf317400 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.562+0000 7fe3b1d83700 20 reqs_thread_entry: start 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.562+0000 7fe3d1614a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.562+0000 7fe3d1614a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.562+0000 7fe3afd7f700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3af55d700 20 reqs_thread_entry: start 2024-04-27T04:34:48.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317c00 con 0x55aabf1bb000 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 20 rados_obj.operate() r=0 bl.length=385 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabee29800 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 170+0+15 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.563+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf1bb000 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.564+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 p5YnriCv1nAtykxBrupQ [delete] v31'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.564+0000 7fe3d1614a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.564+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94191839923584 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361000 con 0x55aabf1bbc00 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94191838331904 notify 133143986183 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55aabf2ddba0 con 0x55aabf1bbc00 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3c9db3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986183 cookie 94191838331904 notifier 4604 bl.length()=170 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(65 notify.2 [notify cookie 94191839923584 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3c9db3700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361400 con 0x55aabf1bbc00 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 94191839923584 notify 133143986183 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55aabf2ddd40 con 0x55aabf1bbc00 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3d1614a80 10 removing email index: 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3d1614a80 10 removing user buckets index 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.565+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf361800 con 0x55aabee29800 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(67 testx$tenanteduser.buckets [delete] v31'3 uv3 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3d1614a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-04-27T04:34:48.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [notify cookie 94191839923584 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360000 con 0x55aabee29800 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94191838334208 notify 133143986189 ret 0) v3 ==== 217+0+0 (crc 0 0 0) 0x55aabf2dba00 con 0x55aabee29800 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3c9db3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986189 cookie 94191838334208 notifier 4604 bl.length()=175 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [notify cookie 94191839923584 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.566+0000 7fe3c9db3700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360400 con 0x55aabee29800 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.567+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== watch-notify(notify_complete (2) cookie 94191839923584 notify 133143986189 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55aabf2d9040 con 0x55aabee29800 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.567+0000 7fe3d1614a80 10 removing user index: testx$tenanteduser 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.567+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(69 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.567+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360800 con 0x55aabf1bb000 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 testx$tenanteduser [call,call,delete] v31'8 uv8 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3d1614a80 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3d1614a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [notify cookie 94191839923584 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e400 con 0x55aabf1bb400 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== watch-notify(notify (1) cookie 94191838333056 notify 133143986194 ret 0) v3 ==== 209+0+0 (crc 0 0 0) 0x55aabf2d5040 con 0x55aabf1bb400 2024-04-27T04:34:48.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3c9db3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986194 cookie 94191838333056 notifier 4604 bl.length()=167 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(71 notify.3 [notify cookie 94191839923584 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.568+0000 7fe3c9db3700 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf360c00 con 0x55aabf1bb400 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.569+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify_complete (2) cookie 94191839923584 notify 133143986194 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55aabf2d51e0 con 0x55aabf1bb400 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.569+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(72 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.569+0000 7fe3b658c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.569+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94191839920128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabee29800 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.570+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94191839920128] v31'144 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.570+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94191838329600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317800 con 0x55aabf1bb400 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.571+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94191838329600] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.571+0000 7fe3d1614a80 20 remove_watcher() i=0 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.571+0000 7fe3d1614a80 2 removed watcher, disabling cache 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.571+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94191838330752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317000 con 0x55aabee29800 2024-04-27T04:34:48.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.572+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94191838330752] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.573+0000 7fe3d1614a80 20 remove_watcher() i=1 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.573+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94191838331904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf316400 con 0x55aabf1bbc00 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.574+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3250898744 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94191838331904] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273200 con 0x55aabf1bbc00 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.574+0000 7fe3d1614a80 20 remove_watcher() i=2 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.574+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94191838333056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf317c00 con 0x55aabf1bb400 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.575+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94191838333056] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf273440 con 0x55aabf1bb400 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.575+0000 7fe3d1614a80 20 remove_watcher() i=3 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.575+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94191838334208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27fc00 con 0x55aabee29800 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.576+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94191838334208] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabee29800 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.576+0000 7fe3d1614a80 20 remove_watcher() i=4 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.576+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94191838335360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27e400 con 0x55aabf1bb000 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.576+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94191838335360] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.577+0000 7fe3d1614a80 20 remove_watcher() i=5 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.577+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94191838336512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27f800 con 0x55aabf1bb000 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.577+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94191838336512] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.577+0000 7fe3d1614a80 20 remove_watcher() i=6 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.577+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94191838337664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aabf27ec00 con 0x55aabf1bb000 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.578+0000 7fe3cfdbf700 1 -- 172.21.15.191:0/3250898744 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94191838337664] v31'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aabf272b40 con 0x55aabf1bb000 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.578+0000 7fe3d1614a80 20 remove_watcher() i=7 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf1bb000 msgr2=0x55aabf113180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf1bb000 0x55aabf113180 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabee29800 msgr2=0x55aabf111b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabee29800 0x55aabf111b80 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf1bb400 msgr2=0x55aabf112c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf1bb400 0x55aabf112c00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf1bbc00 msgr2=0x55aabf112100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf1bbc00 0x55aabf112100 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1bac00 msgr2=0x55aabf111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1bac00 0x55aabf111600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55aabf10b890 tx=0x55aabef19800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 msgr2=0x55aabf110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3250898744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee29400 0x55aabf110b00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55aabfca6750 tx=0x55aabf174900 comp rx=0 tx=0).stop 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 reap_dead start 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3cedbd700 1 -- 172.21.15.191:0/3250898744 reap_dead start 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 shutdown_connections 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 >> 172.21.15.191:0/3250898744 conn(0x55aabee28c00 msgr2=0x55aabef15800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 shutdown_connections 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3250898744 wait complete. 2024-04-27T04:34:48.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf27e800 msgr2=0x55aabf332580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55aabf27e800 0x55aabf332580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabf27e000 msgr2=0x55aabf332000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55aabf27e000 0x55aabf332000 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf317400 msgr2=0x55aabf333080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55aabf317400 0x55aabf333080 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf316800 msgr2=0x55aabf332b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55aabf316800 0x55aabf332b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf316000 msgr2=0x55aabf113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf316000 0x55aabf113700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55aabf2118c0 tx=0x55aabf29d320 comp rx=0 tx=0).stop 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3cf5be700 1 -- 172.21.15.191:0/3372351259 reap_dead start 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 msgr2=0x55aabf112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55aabf290630 tx=0x55aabef198c0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 shutdown_connections 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/3372351259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabf27f400 0x55aabf112680 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 >> 172.21.15.191:0/3372351259 conn(0x55aabf27f000 msgr2=0x55aabef16000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 shutdown_connections 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.579+0000 7fe3d1614a80 1 -- 172.21.15.191:0/3372351259 wait complete. 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1ba000 msgr2=0x55aabf111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1ba000 0x55aabf111080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55aabfca7020 tx=0x55aabf174240 comp rx=0 tx=0).stop 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 msgr2=0x55aabf110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55aabfbf36b0 tx=0x55aabef19a40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 shutdown_connections 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55aabf1ba000 0x55aabf111080 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 --2- 172.21.15.191:0/799238345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aabee28800 0x55aabf110580 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 >> 172.21.15.191:0/799238345 conn(0x55aabee28400 msgr2=0x55aabef15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 shutdown_connections 2024-04-27T04:34:48.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.580+0000 7fe3d1614a80 1 -- 172.21.15.191:0/799238345 wait complete. 2024-04-27T04:34:48.605 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-27T04:34:48.608 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-04-27T04:34:48.651 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:34:48.651 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-27T04:34:48.651 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-27T04:34:48.651 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-04-27T04:34:48.699 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.699 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95c0b36a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95c0b36a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55dfec981500 con 0x55dfec656c00 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95c0b36a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55dfec6f8a00 con 0x55dfec656800 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95beae0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95beae0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47842/0 (socket says 172.21.15.191:47842) 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95be2df700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55dfec656c00 0x55dfec742800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60532/0 (socket says 172.21.15.191:60532) 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.698+0000 7f95beae0700 1 -- 172.21.15.191:0/3360379335 learned_addr learned my addr 172.21.15.191:0/3360379335 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95beae0700 1 -- 172.21.15.191:0/3360379335 >> v1:172.21.15.138:6789/0 conn(0x55dfec656c00 legacy=0x55dfec742800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95beae0700 1 -- 172.21.15.191:0/3360379335 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67fd40 con 0x55dfec656800 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95beae0700 1 --2- 172.21.15.191:0/3360379335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55dfec9399e0 tx=0x55dfec747920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be924aa4f27ed35d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95bdade700 1 -- 172.21.15.191:0/3360379335 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f4a0 con 0x55dfec656800 2024-04-27T04:34:48.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95bdade700 1 -- 172.21.15.191:0/3360379335 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfec981800 con 0x55dfec656800 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95bdade700 1 -- 172.21.15.191:0/3360379335 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f600 con 0x55dfec656800 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3360379335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 msgr2=0x55dfec93e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3360379335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55dfec9399e0 tx=0x55dfec747920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3360379335 shutdown_connections 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3360379335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3360379335 >> 172.21.15.191:0/3360379335 conn(0x55dfec656400 msgr2=0x55dfec742000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3360379335 shutdown_connections 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.699+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3360379335 wait complete. 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.700+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.700+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.700+0000 7f95c0b36a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.700+0000 7f95c0b36a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfec6f8500 con 0x55dfec656800 2024-04-27T04:34:48.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95beae0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95beae0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47858/0 (socket says 172.21.15.191:47858) 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95beae0700 1 -- 172.21.15.191:0/1784492858 learned_addr learned my addr 172.21.15.191:0/1784492858 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95beae0700 1 -- 172.21.15.191:0/1784492858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67fba0 con 0x55dfec656800 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95beae0700 1 --2- 172.21.15.191:0/1784492858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55dfec938b70 tx=0x55dfec747800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8e72ecdbfff135 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95bd2dd700 1 -- 172.21.15.191:0/1784492858 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f760 con 0x55dfec656800 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95bd2dd700 1 -- 172.21.15.191:0/1784492858 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfec981b00 con 0x55dfec656800 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95bd2dd700 1 -- 172.21.15.191:0/1784492858 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f1e0 con 0x55dfec656800 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 -- 172.21.15.191:0/1784492858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 msgr2=0x55dfec93e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/1784492858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55dfec938b70 tx=0x55dfec747800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 -- 172.21.15.191:0/1784492858 shutdown_connections 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/1784492858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 -- 172.21.15.191:0/1784492858 >> 172.21.15.191:0/1784492858 conn(0x55dfec656400 msgr2=0x55dfec743000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 -- 172.21.15.191:0/1784492858 shutdown_connections 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.701+0000 7f95c0b36a80 1 -- 172.21.15.191:0/1784492858 wait complete. 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95c0b36a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95c0b36a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfec6f83c0 con 0x55dfec656800 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95beae0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95beae0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47864/0 (socket says 172.21.15.191:47864) 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95beae0700 1 -- 172.21.15.191:0/3033567685 learned_addr learned my addr 172.21.15.191:0/3033567685 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95beae0700 1 -- 172.21.15.191:0/3033567685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67fa00 con 0x55dfec656800 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.702+0000 7f95beae0700 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55dfec7c7890 tx=0x55dfec747920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95bbada700 1 -- 172.21.15.191:0/3033567685 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f4a0 con 0x55dfec656800 2024-04-27T04:34:48.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95bbada700 1 -- 172.21.15.191:0/3033567685 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfec981c80 con 0x55dfec656800 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dfec67f860 con 0x55dfec656800 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95bbada700 1 -- 172.21.15.191:0/3033567685 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f600 con 0x55dfec656800 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfec67f6c0 con 0x55dfec656800 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95c0b36a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95c0b36a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfec692640 con 0x55dfec657400 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95bbada700 1 -- 172.21.15.191:0/3033567685 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55dfed3f8b00 con 0x55dfec656800 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.703+0000 7f95bf2e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95bf2e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47880/0 (socket says 172.21.15.191:47880) 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95bf2e1700 1 -- 172.21.15.191:0/949140235 learned_addr learned my addr 172.21.15.191:0/949140235 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95bbada700 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8000 0x55dfec93f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.725 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95bbada700 1 -- 172.21.15.191:0/3033567685 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55dfec6d1180 con 0x55dfec656800 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95bf2e1700 1 -- 172.21.15.191:0/949140235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67e000 con 0x55dfec657400 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95beae0700 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8000 0x55dfec93f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/949140235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55dfed4e70b0 tx=0x55dfec747ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d555f023a4bd0014 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95b92d5700 1 -- 172.21.15.191:0/949140235 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f4a0 con 0x55dfec657400 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95b92d5700 1 -- 172.21.15.191:0/949140235 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfec6f7c80 con 0x55dfec657400 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95b92d5700 1 -- 172.21.15.191:0/949140235 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f600 con 0x55dfec657400 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95c0b36a80 1 -- 172.21.15.191:0/949140235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 msgr2=0x55dfec93eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95beae0700 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8000 0x55dfec93f080 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55dfed4e69c0 tx=0x55dfec9aac60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/949140235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55dfed4e70b0 tx=0x55dfec747ce0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95c0b36a80 1 -- 172.21.15.191:0/949140235 shutdown_connections 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/949140235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95c0b36a80 1 -- 172.21.15.191:0/949140235 >> 172.21.15.191:0/949140235 conn(0x55dfec656c00 msgr2=0x55dfec743800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.704+0000 7f95c0b36a80 1 -- 172.21.15.191:0/949140235 shutdown_connections 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95c0b36a80 1 -- 172.21.15.191:0/949140235 wait complete. 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95c0b36a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95c0b36a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfec6928c0 con 0x55dfec657400 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95bf2e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95bf2e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47892/0 (socket says 172.21.15.191:47892) 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 learned_addr learned my addr 172.21.15.191:0/3311017394 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67eea0 con 0x55dfec657400 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.705+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55dfed4e6420 tx=0x55dfec747bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95b7ad2700 1 -- 172.21.15.191:0/3311017394 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f760 con 0x55dfec657400 2024-04-27T04:34:48.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95b7ad2700 1 -- 172.21.15.191:0/3311017394 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfec6f7b00 con 0x55dfec657400 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dfec67e820 con 0x55dfec657400 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95b7ad2700 1 -- 172.21.15.191:0/3311017394 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfec70f4a0 con 0x55dfec657400 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfec67e9c0 con 0x55dfec657400 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95b7ad2700 1 -- 172.21.15.191:0/3311017394 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55dfed3f8dc0 con 0x55dfec657400 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95b7ad2700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8c00 0x55dfec93f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.706+0000 7f95b7ad2700 1 -- 172.21.15.191:0/3311017394 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55dfec6d1340 con 0x55dfec657400 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95be2df700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8c00 0x55dfec93f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfec657800 0x55dfec93fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec657c00 con 0x55dfec657800 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfec657800 0x55dfec93fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95be2df700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8c00 0x55dfec93f600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55dfec939f80 tx=0x55dfeca860c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfec657800 0x55dfec93fb80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 20 realm 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.707+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9000 con 0x55dfec657800 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9400 con 0x55dfec657800 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.729 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9800 con 0x55dfec657800 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfec9e9c00 0x55dfec940100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9000 con 0x55dfec9e9c00 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.708+0000 7f95beae0700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfec9e9c00 0x55dfec940100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95beae0700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfec9e9c00 0x55dfec940100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95c0b36a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95c0b36a80 20 searching for the correct realm 2024-04-27T04:34:48.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9400 con 0x55dfec9e9c00 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95beae0700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfec9e9400 0x55dfec940c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9000 con 0x55dfec9e9400 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.709+0000 7f95be2df700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfec9e9400 0x55dfec940c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.710+0000 7f95be2df700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfec9e9400 0x55dfec940c00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.710+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.710+0000 7f95be2df700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfec9e9000 0x55dfec941180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.710+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.710+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfec9e9000 0x55dfec941180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.710+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfec9e9000 0x55dfec941180 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.711+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9400 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.712+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9400 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.713+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.714+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9400 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.715+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9400 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.716+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9c00 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9400 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95c0b36a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95c0b36a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95c0b36a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95c0b36a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.717+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec657c00 con 0x55dfec657800 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac400 con 0x55dfec657800 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac800 con 0x55dfec9e9c00 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 zone default found 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 4 Realm: () 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 zonegroup default 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaacc00 con 0x55dfec9e9000 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.718+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad000 con 0x55dfec9e9c00 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad400 con 0x55dfec9e9c00 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 20 started sync module instance, tier type = 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.719+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfec9e9400 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.720+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.720+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94420235148032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac800 con 0x55dfec9e9400 2024-04-27T04:34:48.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.720+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad000 con 0x55dfec657800 2024-04-27T04:34:48.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.721+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94420235148032] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.721+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.721+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94420235149184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad400 con 0x55dfec657800 2024-04-27T04:34:48.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.721+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac400 con 0x55dfec9e9c00 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.722+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94420235149184] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.722+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.722+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94420235150336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaacc00 con 0x55dfec9e9c00 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.722+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec657c00 con 0x55dfec9e9400 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.723+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94420235150336] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.723+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.723+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94420235151488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec9e9800 con 0x55dfec9e9400 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.723+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec657800 2024-04-27T04:34:48.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.724+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94420235151488] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.724+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.724+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94420235152640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfec657800 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.724+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac800 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.725+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.725+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94420235153792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaadc00 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.725+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94420235152640] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.725+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad000 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.726+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.726+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94420235154944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad400 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.726+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac000 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.726+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94420235153792] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94420235154944] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94420235156096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfec9e9000 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=0 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=1 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=2 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=3 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=4 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=5 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.727+0000 7f95c0b36a80 20 add_watcher() i=6 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.728+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94420235156096] v31'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.728+0000 7f95c0b36a80 20 add_watcher() i=7 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.728+0000 7f95c0b36a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.728+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac800 con 0x55dfec657800 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.728+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.728+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94420236713984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaadc00 con 0x55dfec657800 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.729+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94420236713984] v31'145 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.729+0000 7f95a5aae700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95c0b36a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95c0b36a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95c0b36a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95c0b36a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfec6f83c0 con 0x55dfecaad400 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95be2df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95be2df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47906/0 (socket says 172.21.15.191:47906) 2024-04-27T04:34:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95be2df700 1 -- 172.21.15.191:0/543522417 learned_addr learned my addr 172.21.15.191:0/543522417 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95be2df700 1 -- 172.21.15.191:0/543522417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67eb60 con 0x55dfecaad400 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.730+0000 7f95be2df700 1 --2- 172.21.15.191:0/543522417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55dfeca3f770 tx=0x55dfec9abf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0547dc2b4caaa60 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95a52ad700 1 -- 172.21.15.191:0/543522417 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfecacc000 con 0x55dfecaad400 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95a52ad700 1 -- 172.21.15.191:0/543522417 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfecad4000 con 0x55dfecaad400 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95a52ad700 1 -- 172.21.15.191:0/543522417 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfecacc160 con 0x55dfecaad400 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 -- 172.21.15.191:0/543522417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 msgr2=0x55dfec940680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/543522417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55dfeca3f770 tx=0x55dfec9abf80 comp rx=0 tx=0).stop 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 -- 172.21.15.191:0/543522417 shutdown_connections 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/543522417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 -- 172.21.15.191:0/543522417 >> 172.21.15.191:0/543522417 conn(0x55dfecaad000 msgr2=0x55dfec744000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 -- 172.21.15.191:0/543522417 shutdown_connections 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.731+0000 7f95c0b36a80 1 -- 172.21.15.191:0/543522417 wait complete. 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95c0b36a80 1 Processor -- start 2024-04-27T04:34:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95c0b36a80 1 -- start start 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95c0b36a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95c0b36a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfec6f8500 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95be2df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95be2df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47910/0 (socket says 172.21.15.191:47910) 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 learned_addr learned my addr 172.21.15.191:0/2565467756 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfec67ed00 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95be2df700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55dfecac41e0 tx=0x55dfeca864e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95a3aaa700 1 -- 172.21.15.191:0/2565467756 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfecacc2c0 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95a3aaa700 1 -- 172.21.15.191:0/2565467756 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfecad4180 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dfec67e340 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95a3aaa700 1 -- 172.21.15.191:0/2565467756 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55dfecacc000 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.732+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfec67eb60 con 0x55dfecaad400 2024-04-27T04:34:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95a3aaa700 1 -- 172.21.15.191:0/2565467756 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55dfed3f9b80 con 0x55dfecaad400 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95a3aaa700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfecb4e000 0x55dfec941700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95a3aaa700 1 -- 172.21.15.191:0/2565467756 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55dfec6d1500 con 0x55dfecaad400 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfecb4e000 0x55dfec941700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfecaac000 0x55dfecb62000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfecaac000 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.733+0000 7f95beae0700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfecaac000 0x55dfecb62000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.734+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfecb4e000 0x55dfec941700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55dfeca3f140 tx=0x55dfeca86360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.734+0000 7f95beae0700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfecaac000 0x55dfecb62000 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.735+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'50 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecaac000 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.735+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfecaac800 0x55dfecb62580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.735+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4e400 con 0x55dfecaac800 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.735+0000 7f95be2df700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfecaac800 0x55dfecb62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.735+0000 7f95be2df700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfecaac800 0x55dfecb62580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.736+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'23 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.736+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfecb4e800 0x55dfecb62b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.736+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4ec00 con 0x55dfecb4e800 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.736+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfecb4e800 0x55dfecb62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.736+0000 7f95bf2e1700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfecb4e800 0x55dfecb62b00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.737+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.737+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4f000 con 0x55dfecaac000 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.738+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'146 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecaac000 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.738+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfecb4f400 0x55dfecb63080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.738+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4f800 con 0x55dfecb4f400 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.738+0000 7f95beae0700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfecb4f400 0x55dfecb63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.739+0000 7f95beae0700 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfecb4f400 0x55dfecb63080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.739+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'50 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.740+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4fc00 con 0x55dfecb4e800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.740+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'40 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.741+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4f000 con 0x55dfecaac000 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.742+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecaac000 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.742+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac400 con 0x55dfecaac800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.743+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'35 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.743+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaacc00 con 0x55dfecb4f400 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.743+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.743+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfecb4e800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.745+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.745+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96000 con 0x55dfecb4f400 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.746+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'51 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.746+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96400 con 0x55dfecaac800 2024-04-27T04:34:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.747+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.747+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96800 con 0x55dfecb4f400 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.748+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.748+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96c00 con 0x55dfecaac800 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.749+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'49 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.749+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97000 con 0x55dfecaac000 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.750+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'147 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecaac000 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.750+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97400 con 0x55dfecb4e800 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.751+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'41 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.751+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97800 con 0x55dfecb4f400 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.752+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'52 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.752+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97c00 con 0x55dfecb4e800 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.753+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'29 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.754+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96000 con 0x55dfecaac000 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.754+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecaac000 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.754+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96800 con 0x55dfecaac000 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.755+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'148 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecaac000 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.755+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96400 con 0x55dfecb4e800 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.756+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'42 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.756+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaacc00 con 0x55dfecb4f400 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.758+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'33 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.758+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac400 con 0x55dfecb4e800 2024-04-27T04:34:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.759+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.759+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4f800 con 0x55dfecb4e800 2024-04-27T04:34:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.760+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'35 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.760+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4f000 con 0x55dfecb4e800 2024-04-27T04:34:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.761+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.761+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4e400 con 0x55dfecb4e800 2024-04-27T04:34:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.762+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'41 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.762+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec657c00 con 0x55dfecaac800 2024-04-27T04:34:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.763+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.763+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97400 con 0x55dfecb4e800 2024-04-27T04:34:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.764+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'29 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.764+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97c00 con 0x55dfecaac800 2024-04-27T04:34:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.765+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'36 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.765+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96400 con 0x55dfecb4e800 2024-04-27T04:34:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.766+0000 7f95bf2e1700 1 -- 172.21.15.191:0/2565467756 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'43 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfecb4e800 2024-04-27T04:34:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.766+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaadc00 con 0x55dfecaac800 2024-04-27T04:34:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.767+0000 7f95be2df700 1 -- 172.21.15.191:0/2565467756 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'50 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfecaac800 2024-04-27T04:34:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.767+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfecb4f400 2024-04-27T04:34:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.768+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfecb4f400 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95a12a5700 20 reqs_thread_entry: start 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f959f2a1700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:18 2024 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f959ea7f700 20 reqs_thread_entry: start 2024-04-27T04:34:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4fc00 con 0x55dfec9e9400 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv28 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.769+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaadc00 con 0x55dfec9e9000 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 20 get_obj_state: rctx=0x55dfec9d0230 obj=myfoo:four state=0x55dfecc181e8 s->prefetch_data=0 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 3.31s0 3:8f8eaf3e:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaacc00 con 0x55dfec9e9000 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1_four [getxattrs out=851b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 234+0+867 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 10 manifest: total_size = 4 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.13018128423674301384 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 20 get_obj_state: rctx=0x55dfec9d0230 obj=myfoo:four state=0x55dfecc181e8 s->prefetch_data=0 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 20 prepare_atomic_modification: state is not atomic. state=0x55dfecc181e8 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 2024-04-27T04:34:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.770+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaad800 con 0x55dfec657800 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.771+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(65 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [stat,call,call] v31'11 uv11 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.771+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 3.31s0 3:8f8eaf3e:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96c00 con 0x55dfec9e9000 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.773+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(66 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1_four [call] v31'2 uv2 ondisk = 0) v8 ==== 192+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.773+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97000 con 0x55dfec657800 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.774+0000 7f95a5aae700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.774+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94420236713984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97800 con 0x55dfec657800 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.774+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [stat,call,call] v31'12 uv12 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.774+0000 7f95b92d5700 20 handle_completion(): completion ok for obj=four 2024-04-27T04:34:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.775+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 data_loggenerations_metadata [watch unwatch cookie 94420236713984] v31'149 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.775+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94420235148032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96000 con 0x55dfec9e9400 2024-04-27T04:34:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.777+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(69 notify.0 [watch unwatch cookie 94420235148032] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.777+0000 7f95c0b36a80 20 remove_watcher() i=0 2024-04-27T04:34:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.777+0000 7f95c0b36a80 2 removed watcher, disabling cache 2024-04-27T04:34:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.777+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94420235149184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb96800 con 0x55dfec657800 2024-04-27T04:34:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.778+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(70 notify.1 [watch unwatch cookie 94420235149184] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.778+0000 7f95c0b36a80 20 remove_watcher() i=1 2024-04-27T04:34:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.778+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94420235150336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb4fc00 con 0x55dfec9e9c00 2024-04-27T04:34:48.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.779+0000 7f95beae0700 1 -- 172.21.15.191:0/3311017394 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(71 notify.2 [watch unwatch cookie 94420235150336] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b200 con 0x55dfec9e9c00 2024-04-27T04:34:48.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.779+0000 7f95c0b36a80 20 remove_watcher() i=2 2024-04-27T04:34:48.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.779+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94420235151488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfec657c00 con 0x55dfec9e9400 2024-04-27T04:34:48.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.780+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(72 notify.3 [watch unwatch cookie 94420235151488] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9b440 con 0x55dfec9e9400 2024-04-27T04:34:48.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.780+0000 7f95c0b36a80 20 remove_watcher() i=3 2024-04-27T04:34:48.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.780+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94420235152640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaac400 con 0x55dfec657800 2024-04-27T04:34:48.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.781+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(73 notify.4 [watch unwatch cookie 94420235152640] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec657800 2024-04-27T04:34:48.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.781+0000 7f95c0b36a80 20 remove_watcher() i=4 2024-04-27T04:34:48.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.781+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94420235153792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaadc00 con 0x55dfec9e9000 2024-04-27T04:34:48.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.782+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(74 notify.5 [watch unwatch cookie 94420235153792] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.782+0000 7f95c0b36a80 20 remove_watcher() i=5 2024-04-27T04:34:48.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.782+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94420235154944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecaacc00 con 0x55dfec9e9000 2024-04-27T04:34:48.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.783+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(75 notify.6 [watch unwatch cookie 94420235154944] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.783+0000 7f95c0b36a80 20 remove_watcher() i=6 2024-04-27T04:34:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.783+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94420235156096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dfecb97400 con 0x55dfec9e9000 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95bf2e1700 1 -- 172.21.15.191:0/3311017394 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(76 notify.7 [watch unwatch cookie 94420235156096] v31'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55dfeca9ab40 con 0x55dfec9e9000 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 20 remove_watcher() i=7 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfec9e9000 msgr2=0x55dfec941180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfec9e9000 0x55dfec941180 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfec657800 msgr2=0x55dfec93fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfec657800 0x55dfec93fb80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfec9e9400 msgr2=0x55dfec940c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfec9e9400 0x55dfec940c00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfec9e9c00 msgr2=0x55dfec940100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfec9e9c00 0x55dfec940100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8c00 msgr2=0x55dfec93f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8c00 0x55dfec93f600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55dfec939f80 tx=0x55dfeca860c0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 msgr2=0x55dfec93eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3311017394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec657400 0x55dfec93eb00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55dfed4e6420 tx=0x55dfec747bc0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 reap_dead start 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 shutdown_connections 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 >> 172.21.15.191:0/3311017394 conn(0x55dfec656c00 msgr2=0x55dfec743800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95be2df700 1 -- 172.21.15.191:0/3311017394 reap_dead start 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 shutdown_connections 2024-04-27T04:34:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.784+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3311017394 wait complete. 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfecaac800 msgr2=0x55dfecb62580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55dfecaac800 0x55dfecb62580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfecaac000 msgr2=0x55dfecb62000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55dfecaac000 0x55dfecb62000 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfecb4f400 msgr2=0x55dfecb63080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55dfecb4f400 0x55dfecb63080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfecb4e800 msgr2=0x55dfecb62b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55dfecb4e800 0x55dfecb62b00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfecb4e000 msgr2=0x55dfec941700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfecb4e000 0x55dfec941700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55dfeca3f140 tx=0x55dfeca86360 comp rx=0 tx=0).stop 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 msgr2=0x55dfec940680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95beae0700 1 -- 172.21.15.191:0/2565467756 reap_dead start 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55dfecac41e0 tx=0x55dfeca864e0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 shutdown_connections 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/2565467756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfecaad400 0x55dfec940680 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 >> 172.21.15.191:0/2565467756 conn(0x55dfecaad000 msgr2=0x55dfec744000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 shutdown_connections 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/2565467756 wait complete. 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8000 msgr2=0x55dfec93f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8000 0x55dfec93f080 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55dfed4e69c0 tx=0x55dfec9aac60 comp rx=0 tx=0).stop 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 msgr2=0x55dfec93e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55dfec7c7890 tx=0x55dfec747920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 shutdown_connections 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55dfec9e8000 0x55dfec93f080 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 --2- 172.21.15.191:0/3033567685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfec656800 0x55dfec93e580 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 >> 172.21.15.191:0/3033567685 conn(0x55dfec656400 msgr2=0x55dfec743000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.785+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 shutdown_connections 2024-04-27T04:34:48.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.786+0000 7f95c0b36a80 1 -- 172.21.15.191:0/3033567685 wait complete. 2024-04-27T04:34:48.793 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T04:34:48.793 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T04:34:48.793 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T04:34:48.837 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.837 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e651a3a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x557af532ec00 0x557af5616580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e651a3a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557af5659500 con 0x557af532e800 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e651a3a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x557af53d0a00 con 0x557af532ec00 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e6294c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x557af532ec00 0x557af5616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e6314d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x557af532e800 0x557af541a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60548/0 (socket says 172.21.15.191:60548) 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e6314d700 1 -- 172.21.15.191:0/3844496539 learned_addr learned my addr 172.21.15.191:0/3844496539 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e6294c700 1 -- 172.21.15.191:0/3844496539 >> v1:172.21.15.138:6789/0 conn(0x557af532e800 legacy=0x557af541a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e6294c700 1 -- 172.21.15.191:0/3844496539 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5357d40 con 0x557af532ec00 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.837+0000 7f2e6294c700 1 --2- 172.21.15.191:0/3844496539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532ec00 0x557af5616580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x557af5611890 tx=0x557af541f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397f41a93d6375c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e6214b700 1 -- 172.21.15.191:0/3844496539 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e74a0 con 0x557af532ec00 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e6214b700 1 -- 172.21.15.191:0/3844496539 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af5659800 con 0x557af532ec00 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e6214b700 1 -- 172.21.15.191:0/3844496539 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e7600 con 0x557af532ec00 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3844496539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532ec00 msgr2=0x557af5616580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3844496539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532ec00 0x557af5616580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x557af5611890 tx=0x557af541f920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3844496539 shutdown_connections 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3844496539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532ec00 0x557af5616580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3844496539 >> 172.21.15.191:0/3844496539 conn(0x557af532e400 msgr2=0x557af541a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3844496539 shutdown_connections 2024-04-27T04:34:48.841 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.838+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3844496539 wait complete. 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e651a3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e651a3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af53d0500 con 0x557af532e800 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e6314d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e6314d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47914/0 (socket says 172.21.15.191:47914) 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.839+0000 7f2e6314d700 1 -- 172.21.15.191:0/3864182276 learned_addr learned my addr 172.21.15.191:0/3864182276 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e6314d700 1 -- 172.21.15.191:0/3864182276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5357ba0 con 0x557af532e800 2024-04-27T04:34:48.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e6314d700 1 --2- 172.21.15.191:0/3864182276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x557af61938c0 tx=0x557af541fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0376e4b8d79ea64 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e6194a700 1 -- 172.21.15.191:0/3864182276 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e74a0 con 0x557af532e800 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e6194a700 1 -- 172.21.15.191:0/3864182276 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af5659b00 con 0x557af532e800 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e6194a700 1 -- 172.21.15.191:0/3864182276 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e7600 con 0x557af532e800 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3864182276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 msgr2=0x557af5616580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3864182276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x557af61938c0 tx=0x557af541fb60 comp rx=0 tx=0).stop 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3864182276 shutdown_connections 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3864182276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3864182276 >> 172.21.15.191:0/3864182276 conn(0x557af532e400 msgr2=0x557af541b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3864182276 shutdown_connections 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.840+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3864182276 wait complete. 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e651a3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e651a3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af53d03c0 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e6314d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e6314d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47922/0 (socket says 172.21.15.191:47922) 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e6314d700 1 -- 172.21.15.191:0/3258404021 learned_addr learned my addr 172.21.15.191:0/3258404021 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.841+0000 7f2e6314d700 1 -- 172.21.15.191:0/3258404021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5357a00 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e6314d700 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x557af6193b00 tx=0x557af541fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e60147700 1 -- 172.21.15.191:0/3258404021 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e78c0 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e60147700 1 -- 172.21.15.191:0/3258404021 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af53cfc80 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557af5357860 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e60147700 1 -- 172.21.15.191:0/3258404021 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e74a0 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557af53576c0 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.842+0000 7f2e60147700 1 -- 172.21.15.191:0/3258404021 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x557af60d0b00 con 0x557af532e800 2024-04-27T04:34:48.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e651a3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e651a3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af53d0000 con 0x557af532f400 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e60147700 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8000 0x557af5617080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e60147700 1 -- 172.21.15.191:0/3258404021 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x557af53a9180 con 0x557af532e800 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6394e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6394e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47928/0 (socket says 172.21.15.191:47928) 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6394e700 1 -- 172.21.15.191:0/1852163522 learned_addr learned my addr 172.21.15.191:0/1852163522 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6314d700 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8000 0x557af5617080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6394e700 1 -- 172.21.15.191:0/1852163522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5356000 con 0x557af532f400 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6314d700 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8000 0x557af5617080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x557af61a1020 tx=0x557af5674240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e6394e700 1 --2- 172.21.15.191:0/1852163522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x557af61a0bd0 tx=0x557af5674de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b4a1b0bef718c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e5d942700 1 -- 172.21.15.191:0/1852163522 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e78c0 con 0x557af532f400 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e5d942700 1 -- 172.21.15.191:0/1852163522 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af5658300 con 0x557af532f400 2024-04-27T04:34:48.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.843+0000 7f2e5d942700 1 -- 172.21.15.191:0/1852163522 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e74a0 con 0x557af532f400 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1852163522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 msgr2=0x557af5616b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1852163522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x557af61a0bd0 tx=0x557af5674de0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1852163522 shutdown_connections 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1852163522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1852163522 >> 172.21.15.191:0/1852163522 conn(0x557af532ec00 msgr2=0x557af541b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1852163522 shutdown_connections 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1852163522 wait complete. 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e651a3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af536a640 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e6394e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e6394e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47930/0 (socket says 172.21.15.191:47930) 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.844+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 learned_addr learned my addr 172.21.15.191:0/1094057586 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5356ea0 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e6394e700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x557af61a0750 tx=0x557af541ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e5c13f700 1 -- 172.21.15.191:0/1094057586 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e7b80 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557af5356820 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e5c13f700 1 -- 172.21.15.191:0/1094057586 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af53cfc80 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e5c13f700 1 -- 172.21.15.191:0/1094057586 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e78c0 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.845+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557af53569c0 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e5c13f700 1 -- 172.21.15.191:0/1094057586 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x557af60d0dc0 con 0x557af532f400 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e5c13f700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8c00 0x557af5617600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e5c13f700 1 -- 172.21.15.191:0/1094057586 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x557af53a9340 con 0x557af532f400 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e6294c700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8c00 0x557af5617600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557af532f800 0x557af5617b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af532fc00 con 0x557af532f800 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e6394e700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557af532f800 0x557af5617b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e6294c700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8c00 0x557af5617600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x557af5611890 tx=0x557af541f800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.846+0000 7f2e6394e700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557af532f800 0x557af5617b80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 realm 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9000 con 0x557af532f800 2024-04-27T04:34:48.865 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9400 con 0x557af532f800 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9800 con 0x557af532f800 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.847+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557af56b9c00 0x557af5618100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9000 con 0x557af56b9c00 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e6314d700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557af56b9c00 0x557af5618100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e6314d700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557af56b9c00 0x557af5618100 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e651a3a80 20 searching for the correct realm 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.848+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9400 con 0x557af56b9c00 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6314d700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557af56b9400 0x557af5618c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9000 con 0x557af56b9400 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6294c700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557af56b9400 0x557af5618c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6294c700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557af56b9400 0x557af5618c00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6294c700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557af56b9000 0x557af5619180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.849+0000 7f2e6394e700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557af56b9000 0x557af5619180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557af56b9000 0x557af5619180 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.850+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9400 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.851+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9400 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.852+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.853+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9400 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.854+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.855+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9400 2024-04-27T04:34:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9c00 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9400 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e651a3a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e651a3a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e651a3a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e651a3a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.856+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af532fc00 con 0x557af532f800 2024-04-27T04:34:48.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788400 con 0x557af532f800 2024-04-27T04:34:48.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788800 con 0x557af56b9c00 2024-04-27T04:34:48.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 zone default found 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 4 Realm: () 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 zonegroup default 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.857+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788c00 con 0x557af56b9000 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789000 con 0x557af56b9c00 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789400 con 0x557af56b9c00 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 started sync module instance, tier type = 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.858+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789800 con 0x557af56b9400 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.859+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.859+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93986591136512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788800 con 0x557af56b9400 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.859+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789000 con 0x557af532f800 2024-04-27T04:34:48.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.860+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.860+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93986591136512] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.860+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93986591137664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789400 con 0x557af532f800 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.861+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788400 con 0x557af56b9c00 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.861+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93986591137664] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.862+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.862+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93986591138816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788c00 con 0x557af56b9c00 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.862+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af532fc00 con 0x557af56b9400 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.862+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93986591138816] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.864+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.864+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93986591139968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af56b9800 con 0x557af56b9400 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.864+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af532f800 2024-04-27T04:34:48.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.865+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93986591139968] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.865+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.865+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93986591141120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789800 con 0x557af532f800 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.865+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788800 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.866+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.866+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93986591142272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789c00 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.866+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789000 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.866+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93986591141120] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.867+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93986591142272] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.867+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.867+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93986591143424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789400 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.867+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788000 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93986591144576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789800 con 0x557af56b9000 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93986591143424] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=0 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=1 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=2 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=3 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=4 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=5 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.868+0000 7f2e651a3a80 20 add_watcher() i=6 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.869+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93986591144576] v31'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.869+0000 7f2e651a3a80 20 add_watcher() i=7 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.869+0000 7f2e651a3a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:48.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.869+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788800 con 0x557af532f800 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.870+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.870+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93986592727040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789c00 con 0x557af532f800 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93986592727040] v31'150 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e4a11b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e651a3a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e651a3a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e651a3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.871+0000 7f2e651a3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af536a780 con 0x557af5789400 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e6294c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e6294c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47938/0 (socket says 172.21.15.191:47938) 2024-04-27T04:34:48.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e6294c700 1 -- 172.21.15.191:0/2557600985 learned_addr learned my addr 172.21.15.191:0/2557600985 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e6294c700 1 -- 172.21.15.191:0/2557600985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5356b60 con 0x557af5789400 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e6294c700 1 --2- 172.21.15.191:0/2557600985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x557af579e060 tx=0x557af541f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec68caaff83ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e4991a700 1 -- 172.21.15.191:0/2557600985 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af53e7760 con 0x557af5789400 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e4991a700 1 -- 172.21.15.191:0/2557600985 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af57aa000 con 0x557af5789400 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.872+0000 7f2e4991a700 1 -- 172.21.15.191:0/2557600985 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af57ac000 con 0x557af5789400 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 -- 172.21.15.191:0/2557600985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 msgr2=0x557af5618680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/2557600985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x557af579e060 tx=0x557af541f980 comp rx=0 tx=0).stop 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 -- 172.21.15.191:0/2557600985 shutdown_connections 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/2557600985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 -- 172.21.15.191:0/2557600985 >> 172.21.15.191:0/2557600985 conn(0x557af5789000 msgr2=0x557af541c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 -- 172.21.15.191:0/2557600985 shutdown_connections 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 -- 172.21.15.191:0/2557600985 wait complete. 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 Processor -- start 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.873+0000 7f2e651a3a80 1 -- start start 2024-04-27T04:34:48.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e651a3a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e651a3a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af53d0000 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e6294c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e6294c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47946/0 (socket says 172.21.15.191:47946) 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e6294c700 1 -- 172.21.15.191:0/3782161354 learned_addr learned my addr 172.21.15.191:0/3782161354 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e6294c700 1 -- 172.21.15.191:0/3782161354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af5356d00 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e6294c700 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x557af579e270 tx=0x557af541f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e48117700 1 -- 172.21.15.191:0/3782161354 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af57ac160 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e48117700 1 -- 172.21.15.191:0/3782161354 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557af57aa180 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557af5356340 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e48117700 1 -- 172.21.15.191:0/3782161354 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557af57ac2c0 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.874+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557af5357d40 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.875+0000 7f2e48117700 1 -- 172.21.15.191:0/3782161354 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x557af60d1b80 con 0x557af5789400 2024-04-27T04:34:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.875+0000 7f2e48117700 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af5824000 0x557af5619700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.875+0000 7f2e48117700 1 -- 172.21.15.191:0/3782161354 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x557af53a9500 con 0x557af5789400 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.875+0000 7f2e6394e700 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af5824000 0x557af5619700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.875+0000 7f2e651a3a80 5 note: GC not initialized 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.876+0000 7f2e6394e700 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af5824000 0x557af5619700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x557af5715f80 tx=0x557af57ad320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.876+0000 7f2e45912700 20 reqs_thread_entry: start 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.876+0000 7f2e651a3a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.876+0000 7f2e651a3a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.876+0000 7f2e4390e700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:19 2024 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.876+0000 7f2e430ec700 20 reqs_thread_entry: start 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789800 con 0x557af56b9400 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv28 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788800 con 0x557af56b9000 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.877+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824400 con 0x557af56b9400 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv28 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824800 con 0x557af56b9000 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(65 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 4.25 4:a741c040:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824c00 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 4.39 4:9e5e97cb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825000 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 4.1c 4:381df985:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825400 con 0x557af56b9000 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 4.1c 4:3a1291c0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825800 con 0x557af56b9000 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 4.5 4:a135d554:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825c00 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 4.2a 4:57f23c0a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824c00 con 0x557af532f800 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 4.2a 4:57a4ba66:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825000 con 0x557af532f800 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 4.11 4:8b5d38c9:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788400 con 0x557af56b9000 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:74 4.29 4:95afba01:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788c00 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.2e 4:74385e64:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824800 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.878+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 4.1f 4:fa83b5f2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825400 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(76 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.3 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.5 [call out=124b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1.6 [call out=124b] v0'0 uv12 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e4a11b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.879+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93986592727040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825800 con 0x557af532f800 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.880+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 93986592727040] v31'151 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.880+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93986591136512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824c00 con 0x557af56b9400 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.881+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 93986591136512] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.881+0000 7f2e651a3a80 20 remove_watcher() i=0 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.881+0000 7f2e651a3a80 2 removed watcher, disabling cache 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.881+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93986591137664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825000 con 0x557af532f800 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.882+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 93986591137664] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.882+0000 7f2e651a3a80 20 remove_watcher() i=1 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.882+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93986591138816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5789c00 con 0x557af56b9c00 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.883+0000 7f2e6314d700 1 -- 172.21.15.191:0/1094057586 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 93986591138816] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f200 con 0x557af56b9c00 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.884+0000 7f2e651a3a80 20 remove_watcher() i=2 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.884+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93986591139968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5788800 con 0x557af56b9400 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.884+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 93986591139968] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576f440 con 0x557af56b9400 2024-04-27T04:34:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.884+0000 7f2e651a3a80 20 remove_watcher() i=3 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.884+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93986591141120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825c00 con 0x557af532f800 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.885+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 93986591141120] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af532f800 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.885+0000 7f2e651a3a80 20 remove_watcher() i=4 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.885+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93986591142272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824800 con 0x557af56b9000 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.887+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 93986591142272] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.887+0000 7f2e651a3a80 20 remove_watcher() i=5 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.887+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93986591143424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5825400 con 0x557af56b9000 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.888+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 93986591143424] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.888+0000 7f2e651a3a80 20 remove_watcher() i=6 2024-04-27T04:34:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.888+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93986591144576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557af5824400 con 0x557af56b9000 2024-04-27T04:34:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e6394e700 1 -- 172.21.15.191:0/1094057586 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 93986591144576] v31'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557af576eb40 con 0x557af56b9000 2024-04-27T04:34:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 20 remove_watcher() i=7 2024-04-27T04:34:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557af56b9000 msgr2=0x557af5619180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557af56b9000 0x557af5619180 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557af532f800 msgr2=0x557af5617b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557af532f800 0x557af5617b80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557af56b9400 msgr2=0x557af5618c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557af56b9400 0x557af5618c00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557af56b9c00 msgr2=0x557af5618100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557af56b9c00 0x557af5618100 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8c00 msgr2=0x557af5617600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8c00 0x557af5617600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x557af5611890 tx=0x557af541f800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 msgr2=0x557af5616b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/1094057586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532f400 0x557af5616b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x557af61a0750 tx=0x557af541ff20 comp rx=0 tx=0).stop 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 reap_dead start 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e6294c700 1 -- 172.21.15.191:0/1094057586 reap_dead start 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 shutdown_connections 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 >> 172.21.15.191:0/1094057586 conn(0x557af532ec00 msgr2=0x557af541b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 shutdown_connections 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/1094057586 wait complete. 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af5824000 msgr2=0x557af5619700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af5824000 0x557af5619700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x557af5715f80 tx=0x557af57ad320 comp rx=0 tx=0).stop 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 msgr2=0x557af5618680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x557af579e270 tx=0x557af541f8c0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 shutdown_connections 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af5824000 0x557af5619700 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3782161354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af5789400 0x557af5618680 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.889+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 >> 172.21.15.191:0/3782161354 conn(0x557af5789000 msgr2=0x557af541c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 shutdown_connections 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3782161354 wait complete. 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8000 msgr2=0x557af5617080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8000 0x557af5617080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x557af61a1020 tx=0x557af5674240 comp rx=0 tx=0).stop 2024-04-27T04:34:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 msgr2=0x557af5616580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x557af6193b00 tx=0x557af541fa40 comp rx=0 tx=0).stop 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 shutdown_connections 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557af56b8000 0x557af5617080 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 --2- 172.21.15.191:0/3258404021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af532e800 0x557af5616580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 >> 172.21.15.191:0/3258404021 conn(0x557af532e400 msgr2=0x557af541b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 shutdown_connections 2024-04-27T04:34:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.890+0000 7f2e651a3a80 1 -- 172.21.15.191:0/3258404021 wait complete. 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "num_shards": 11, 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "tenant": "", 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "zonegroup": "29d2c105-c074-4639-b257-47b1060771f9", 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_rule": "default-placement", 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "explicit_placement": { 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "", 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "", 2024-04-27T04:34:48.910 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "" 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "marker": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": "Normal", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "versioned": false, 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "versioning_enabled": false, 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "object_lock_enabled": false, 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_enabled": false, 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "mtime": "2024-04-27T04:34:48.405147Z", 2024-04-27T04:34:48.911 INFO:teuthology.orchestra.run.smithi191.stdout: "creation_time": "2024-04-27T04:34:45.731629Z", 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "usage": { 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "rgw.main": { 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "size": 0, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "size_actual": 0, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "size_utilized": 0, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb": 0, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb_actual": 0, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "size_kb_utilized": 0, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "num_objects": 0 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:48.912 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:48.913 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:48.913 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:48.913 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:48.913 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:48.913 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:48.913 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:34:48.405147Z', 'creation_time': '2024-04-27T04:34:45.731629Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T04:34:48.913 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-04-27T04:34:48.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-04-27T04:34:48.913 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-04-27T04:34:48.956 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:48.956 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:48.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:48.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:48.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d0bf78a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d0bf78a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b87c465500 con 0x55b87c13ac00 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d0bf78a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b87c1dca00 con 0x55b87c13a800 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d09f22700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d09f22700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47956/0 (socket says 172.21.15.191:47956) 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d09f22700 1 -- 172.21.15.191:0/754130416 learned_addr learned my addr 172.21.15.191:0/754130416 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d09721700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55b87c13ac00 0x55b87c226800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60562/0 (socket says 172.21.15.191:60562) 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d09f22700 1 -- 172.21.15.191:0/754130416 >> v1:172.21.15.138:6789/0 conn(0x55b87c13ac00 legacy=0x55b87c226800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.956+0000 7f0d09f22700 1 -- 172.21.15.191:0/754130416 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c163d40 con 0x55b87c13a800 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d09f22700 1 --2- 172.21.15.191:0/754130416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55b87c41d9e0 tx=0x55b87c22b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cee2f67ff1db474 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d08f20700 1 -- 172.21.15.191:0/754130416 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f34a0 con 0x55b87c13a800 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d08f20700 1 -- 172.21.15.191:0/754130416 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c465800 con 0x55b87c13a800 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d08f20700 1 -- 172.21.15.191:0/754130416 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f3600 con 0x55b87c13a800 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/754130416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 msgr2=0x55b87c422580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/754130416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55b87c41d9e0 tx=0x55b87c22b920 comp rx=0 tx=0).stop 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/754130416 shutdown_connections 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/754130416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/754130416 >> 172.21.15.191:0/754130416 conn(0x55b87c13a400 msgr2=0x55b87c226000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/754130416 shutdown_connections 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.957+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/754130416 wait complete. 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.958+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.958+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.958+0000 7f0d0bf78a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.958+0000 7f0d0bf78a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87c1dc500 con 0x55b87c13a800 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.958+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47958/0 (socket says 172.21.15.191:47958) 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d09f22700 1 -- 172.21.15.191:0/912943642 learned_addr learned my addr 172.21.15.191:0/912943642 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d09f22700 1 -- 172.21.15.191:0/912943642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c163ba0 con 0x55b87c13a800 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d09f22700 1 --2- 172.21.15.191:0/912943642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x55b87cf0c1b0 tx=0x55b87c22b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4f874441363a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d0871f700 1 -- 172.21.15.191:0/912943642 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f3760 con 0x55b87c13a800 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d0871f700 1 -- 172.21.15.191:0/912943642 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c465b00 con 0x55b87c13a800 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d0871f700 1 -- 172.21.15.191:0/912943642 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f34a0 con 0x55b87c13a800 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.959+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/912943642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 msgr2=0x55b87c422580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/912943642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x55b87cf0c1b0 tx=0x55b87c22b800 comp rx=0 tx=0).stop 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/912943642 shutdown_connections 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/912943642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/912943642 >> 172.21.15.191:0/912943642 conn(0x55b87c13a400 msgr2=0x55b87c227000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/912943642 shutdown_connections 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/912943642 wait complete. 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d0bf78a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87c1dc3c0 con 0x55b87c13a800 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47966/0 (socket says 172.21.15.191:47966) 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.960+0000 7f0d09f22700 1 -- 172.21.15.191:0/4206639510 learned_addr learned my addr 172.21.15.191:0/4206639510 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d09f22700 1 -- 172.21.15.191:0/4206639510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c163a00 con 0x55b87c13a800 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55b87cf0df20 tx=0x55b87c22b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d06f1c700 1 -- 172.21.15.191:0/4206639510 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f3600 con 0x55b87c13a800 2024-04-27T04:34:48.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d06f1c700 1 -- 172.21.15.191:0/4206639510 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c465c80 con 0x55b87c13a800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b87c163860 con 0x55b87c13a800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d06f1c700 1 -- 172.21.15.191:0/4206639510 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f38c0 con 0x55b87c13a800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.961+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b87c1636c0 con 0x55b87c13a800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d06f1c700 1 -- 172.21.15.191:0/4206639510 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b87cedcb00 con 0x55b87c13a800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d06f1c700 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c13b400 0x55b87c423080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d0bf78a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d0bf78a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87c1dc000 con 0x55b87c13b800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d06f1c700 1 -- 172.21.15.191:0/4206639510 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b87c1b5180 con 0x55b87c13a800 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c13b400 0x55b87c423080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47976/0 (socket says 172.21.15.191:47976) 2024-04-27T04:34:48.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d09f22700 1 -- 172.21.15.191:0/2244315265 learned_addr learned my addr 172.21.15.191:0/2244315265 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d09f22700 1 -- 172.21.15.191:0/2244315265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c162000 con 0x55b87c13b800 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c13b400 0x55b87c423080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55b87cf0de60 tx=0x55b87c48ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d09f22700 1 --2- 172.21.15.191:0/2244315265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55b87cfc6fc0 tx=0x55b87c48ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead1cbf63c079a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.962+0000 7f0d04717700 1 -- 172.21.15.191:0/2244315265 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f3600 con 0x55b87c13b800 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d04717700 1 -- 172.21.15.191:0/2244315265 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c465c80 con 0x55b87c13b800 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d04717700 1 -- 172.21.15.191:0/2244315265 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f38c0 con 0x55b87c13b800 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2244315265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 msgr2=0x55b87c422b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/2244315265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55b87cfc6fc0 tx=0x55b87c48ac60 comp rx=0 tx=0).stop 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2244315265 shutdown_connections 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/2244315265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:48.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2244315265 >> 172.21.15.191:0/2244315265 conn(0x55b87c13ac00 msgr2=0x55b87c227800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2244315265 shutdown_connections 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2244315265 wait complete. 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.963+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d0bf78a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d0bf78a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87c176640 con 0x55b87c13b800 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d09f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47978/0 (socket says 172.21.15.191:47978) 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 learned_addr learned my addr 172.21.15.191:0/4051528170 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c162ea0 con 0x55b87c13b800 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55b87cfc6000 tx=0x55b87c48ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d02f14700 1 -- 172.21.15.191:0/4051528170 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f3760 con 0x55b87c13b800 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d02f14700 1 -- 172.21.15.191:0/4051528170 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c1dbe00 con 0x55b87c13b800 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b87c162820 con 0x55b87c13b800 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d02f14700 1 -- 172.21.15.191:0/4051528170 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c1f34a0 con 0x55b87c13b800 2024-04-27T04:34:48.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.964+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b87c1629c0 con 0x55b87c13b800 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.965+0000 7f0d02f14700 1 -- 172.21.15.191:0/4051528170 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b87cedcdc0 con 0x55b87c13b800 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.965+0000 7f0d02f14700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c4e6c00 0x55b87c423600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.965+0000 7f0d02f14700 1 -- 172.21.15.191:0/4051528170 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b87c1b5340 con 0x55b87c13b800 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.965+0000 7f0d09721700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c4e6c00 0x55b87c423600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.965+0000 7f0d09721700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c4e6c00 0x55b87c423600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55b87c41df80 tx=0x55b87c552000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c13bc00 0x55b87c423b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7000 con 0x55b87c13bc00 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c13bc00 0x55b87c423b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c13bc00 0x55b87c423b80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 20 realm 2024-04-27T04:34:48.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7400 con 0x55b87c13bc00 2024-04-27T04:34:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.966+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7800 con 0x55b87c13bc00 2024-04-27T04:34:48.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:48.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7c00 con 0x55b87c13bc00 2024-04-27T04:34:48.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c4e7000 0x55b87c424100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7400 con 0x55b87c4e7000 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c4e7000 0x55b87c424100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.967+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c4e7000 0x55b87c424100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d0bf78a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d0bf78a80 20 searching for the correct realm 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a000 con 0x55b87c4e7000 2024-04-27T04:34:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c56a000 0x55b87c424c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7400 con 0x55b87c56a000 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d09721700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c56a000 0x55b87c424c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.968+0000 7f0d09721700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c56a000 0x55b87c424c00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d09721700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e7400 0x55b87c425180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e7400 0x55b87c425180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e7400 0x55b87c425180 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.969+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c56a000 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.970+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.971+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.971+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.971+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.971+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.971+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.971+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c56a000 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.972+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:48.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.973+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c56a000 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.974+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:49.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c56a000 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.975+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7400 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c13bc00 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c56a000 2024-04-27T04:34:49.001 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0bf78a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0bf78a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0bf78a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0bf78a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.976+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a400 con 0x55b87c13bc00 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a800 con 0x55b87c13bc00 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56ac00 con 0x55b87c4e7000 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 zone default found 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 4 Realm: () 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 zonegroup default 2024-04-27T04:34:49.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b000 con 0x55b87c4e7400 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.977+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b400 con 0x55b87c4e7000 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a400 con 0x55b87c4e7000 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 20 started sync module instance, tier type = 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.978+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a800 con 0x55b87c56a000 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.979+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.979+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94250847017728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b000 con 0x55b87c56a000 2024-04-27T04:34:49.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.979+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56bc00 con 0x55b87c13bc00 2024-04-27T04:34:49.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.980+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94250847017728] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.980+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.980+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94250847018880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7c00 con 0x55b87c13bc00 2024-04-27T04:34:49.006 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.980+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7800 con 0x55b87c4e7000 2024-04-27T04:34:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.981+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.981+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94250847020032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e7000 2024-04-27T04:34:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.981+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a800 con 0x55b87c56a000 2024-04-27T04:34:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.982+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94250847018880] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.982+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.982+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94250847021184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b000 con 0x55b87c56a000 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.982+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56ac00 con 0x55b87c13bc00 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.982+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94250847020032] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.983+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.983+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94250847022336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b400 con 0x55b87c13bc00 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.983+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a400 con 0x55b87c4e7400 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.983+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94250847021184] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.984+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.984+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94250847023488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e6400 con 0x55b87c4e7400 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.984+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94250847022336] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.984+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7400 2024-04-27T04:34:49.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.985+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.985+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94250847023488] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.985+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94250847024640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572400 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.985+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572800 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94250847024640] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94250847025792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572c00 con 0x55b87c4e7400 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=0 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=1 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=2 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=3 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=4 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=5 2024-04-27T04:34:49.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.986+0000 7f0d0bf78a80 20 add_watcher() i=6 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.987+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94250847025792] v31'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.987+0000 7f0d0bf78a80 20 add_watcher() i=7 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.987+0000 7f0d0bf78a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.987+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c573000 con 0x55b87c13bc00 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.988+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.988+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94250848460800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c573400 con 0x55b87c13bc00 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.988+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94250848460800] v31'152 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0cf0ef0700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:49.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d0bf78a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d0bf78a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d0bf78a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d0bf78a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87c176780 con 0x55b87c573c00 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d09721700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d09721700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47982/0 (socket says 172.21.15.191:47982) 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.989+0000 7f0d09721700 1 -- 172.21.15.191:0/2545666212 learned_addr learned my addr 172.21.15.191:0/2545666212 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.990+0000 7f0d09721700 1 -- 172.21.15.191:0/2545666212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c162b60 con 0x55b87c573c00 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.990+0000 7f0d09721700 1 --2- 172.21.15.191:0/2545666212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55b87c5037d0 tx=0x55b87c48bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d611b908f2d283ac server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.990+0000 7f0cf06ef700 1 -- 172.21.15.191:0/2545666212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c578160 con 0x55b87c573c00 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.990+0000 7f0cf06ef700 1 -- 172.21.15.191:0/2545666212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c594000 con 0x55b87c573c00 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.990+0000 7f0cf06ef700 1 -- 172.21.15.191:0/2545666212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c5782c0 con 0x55b87c573c00 2024-04-27T04:34:49.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2545666212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 msgr2=0x55b87c424680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/2545666212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55b87c5037d0 tx=0x55b87c48bf80 comp rx=0 tx=0).stop 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2545666212 shutdown_connections 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/2545666212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2545666212 >> 172.21.15.191:0/2545666212 conn(0x55b87c573800 msgr2=0x55b87c228000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2545666212 shutdown_connections 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/2545666212 wait complete. 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 Processor -- start 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- start start 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d0bf78a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87c1dc000 con 0x55b87c573c00 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d09721700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d09721700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:47984/0 (socket says 172.21.15.191:47984) 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.991+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 learned_addr learned my addr 172.21.15.191:0/4263162921 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87c162d00 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0d09721700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55b87c57e7e0 tx=0x55b87c552240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0ceeeec700 1 -- 172.21.15.191:0/4263162921 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c578420 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0ceeeec700 1 -- 172.21.15.191:0/4263162921 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87c594180 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b87c162340 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0ceeeec700 1 -- 172.21.15.191:0/4263162921 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b87c578160 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.992+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b87c162b60 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.993+0000 7f0ceeeec700 1 -- 172.21.15.191:0/4263162921 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b87ceddb80 con 0x55b87c573c00 2024-04-27T04:34:49.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.993+0000 7f0ceeeec700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c56bc00 0x55b87c425700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.993+0000 7f0ceeeec700 1 -- 172.21.15.191:0/4263162921 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55b87c1b56c0 con 0x55b87c573c00 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.993+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c56bc00 0x55b87c425700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.993+0000 7f0d0bf78a80 5 note: GC not initialized 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.993+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c56bc00 0x55b87c425700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55b87c4e3740 tx=0x55b87c5532c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0cec6e7700 20 reqs_thread_entry: start 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0d0bf78a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0d0bf78a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0cea6e3700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:19 2024 2024-04-27T04:34:49.032 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0ce9ec1700 20 reqs_thread_entry: start 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e6400 0x55b87c66c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56ac00 con 0x55b87c4e6400 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.994+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e6400 0x55b87c66c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e6400 0x55b87c66c000 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(1 [pgnls start_epoch 31 out=254b] v31'50 uv41 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e6400 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56ac00 con 0x55b87c4e6400 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(2 [pgnls start_epoch 31 out=131b] v31'4 uv4 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e6400 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c56ac00 0x55b87c66d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c56ac00 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.995+0000 7f0d09721700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c56ac00 0x55b87c66d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d09721700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c56ac00 0x55b87c66d600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 [pgnls start_epoch 31 out=295b] v31'35 uv27 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b800 con 0x55b87c4e6400 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(4 [pgnls start_epoch 31 out=172b] v31'23 uv15 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e6400 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c56b800 0x55b87c66db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c4e7800 con 0x55b87c56b800 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.996+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c56b800 0x55b87c66db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.997+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c56b800 0x55b87c66db80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.997+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(5 [pgnls start_epoch 31 out=335b] v31'152 uv29 ondisk = 1) v8 ==== 144+0+335 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c56b800 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.997+0000 7f0d0a723700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c4e7800 0x55b87c66e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.997+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.997+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c4e7800 0x55b87c66e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.997+0000 7f0d09f22700 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c4e7800 0x55b87c66e100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=213b] v31'25 uv25 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(7 [pgnls start_epoch 31 out=254b] v31'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(8 [pgnls start_epoch 31 out=295b] v31'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(9 [pgnls start_epoch 31 out=213b] v31'16 uv16 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.998+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(10 [pgnls start_epoch 31 out=172b] v31'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=336b] v31'33 uv25 ondisk = 1) v8 ==== 144+0+336 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(12 [pgnls start_epoch 31 out=131b] v31'29 uv21 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(13 [pgnls start_epoch 31 out=254b] v31'12 uv12 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(14 [pgnls start_epoch 31 out=90b] v31'29 uv21 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:48.999+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56b800 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(15 [pgnls start_epoch 31 out=213b] v31'25 uv17 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c56b800 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(16 [pgnls start_epoch 31 out=254b] v31'17 uv16 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56b800 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(17 [pgnls start_epoch 31 out=254b] v31'20 uv20 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c56b800 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.000+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(18 [pgnls start_epoch 31 out=213b] v31'25 uv17 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(19 [pgnls start_epoch 31 out=172b] v31'41 uv24 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56b800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=295b] v31'50 uv46 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c56b800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(21 [pgnls start_epoch 31 out=276b] v31'13 uv13 ondisk = 1) v8 ==== 144+0+276 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.001+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.002+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=172b] v31'23 uv15 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.002+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56b800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.002+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(23 [pgnls start_epoch 31 out=295b] v31'25 uv17 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c56b800 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.002+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e6400 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.002+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(24 [pgnls start_epoch 31 out=172b] v31'36 uv20 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e6400 2024-04-27T04:34:49.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.002+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(25 [pgnls start_epoch 31 out=213b] v31'52 uv40 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e6400 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(26 [pgnls start_epoch 31 out=172b] v31'19 uv11 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e6400 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(27 [pgnls start_epoch 31 out=172b] v31'23 uv15 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(28 [pgnls start_epoch 31 out=90b] v31'19 uv11 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.003+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56ac00 2024-04-27T04:34:49.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.004+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(29 [pgnls start_epoch 31 out=90b] v31'43 uv19 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56ac00 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.004+0000 7f0d09721700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c56b800 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.004+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(30 [pgnls start_epoch 31 out=131b] v31'12 uv12 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c56b800 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.004+0000 7f0d0a723700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e7800 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.004+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(31 [pgnls start_epoch 31 out=295b] v31'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7800 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.004+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c572000 con 0x55b87c4e6400 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.005+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(32 [pgnls start_epoch 31 out=213b] v31'25 uv17 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e6400 2024-04-27T04:34:49.039 INFO:teuthology.orchestra.run.smithi191.stdout:[ 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000078", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000070", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000104", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000026", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000028", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000040", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000015", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000069", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000095", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000003", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000047", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000035", 2024-04-27T04:34:49.040 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000037", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000024", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000000", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000031", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "data_loggenerations_metadata", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000076", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000113", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000005", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000011", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000058", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000052", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000088", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000080", 2024-04-27T04:34:49.041 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000090", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000110", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000096", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000087", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000008", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000006", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000029", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000089", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000022", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000016", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000041", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000018", 2024-04-27T04:34:49.042 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000092", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000014", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000112", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000007", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000021", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000064", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000071", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000074", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000081", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000009", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000121", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000125", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000082", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000105", 2024-04-27T04:34:49.043 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000059", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000077", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000032", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000053", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000091", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000065", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000083", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000010", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000045", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000002", 2024-04-27T04:34:49.044 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000116", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000034", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000101", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000079", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000049", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000117", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000044", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000066", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000068", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000085", 2024-04-27T04:34:49.045 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000073", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000038", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000118", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000036", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000103", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000119", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000098", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000027", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000019", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000039", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000100", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000093", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000004", 2024-04-27T04:34:49.046 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000063", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000122", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000057", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000054", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000114", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "data_log.0", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000012", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000084", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000043", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000111", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000048", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000020", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000099", 2024-04-27T04:34:49.047 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000056", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000072", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000062", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000115", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000033", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000025", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000123", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000108", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000094", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000060", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000109", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000013", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000051", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000106", 2024-04-27T04:34:49.048 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000017", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000023", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000107", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000046", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000067", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000086", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000050", 2024-04-27T04:34:49.049 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000102", 2024-04-27T04:34:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000124", 2024-04-27T04:34:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000001", 2024-04-27T04:34:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000126", 2024-04-27T04:34:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000055", 2024-04-27T04:34:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000061", 2024-04-27T04:34:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000097", 2024-04-27T04:34:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000042", 2024-04-27T04:34:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000075", 2024-04-27T04:34:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000120", 2024-04-27T04:34:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: "obj_delete_at_hint.0000000030" 2024-04-27T04:34:49.051 INFO:teuthology.orchestra.run.smithi191.stdout:] 2024-04-27T04:34:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.005+0000 7f0cf0ef0700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.006+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94250848460800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56b400 con 0x55b87c13bc00 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.007+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94250848460800] v31'153 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.007+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94250847017728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c56a400 con 0x55b87c56a000 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.008+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94250847017728] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.008+0000 7f0d0bf78a80 20 remove_watcher() i=0 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.008+0000 7f0d0bf78a80 2 removed watcher, disabling cache 2024-04-27T04:34:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.008+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94250847018880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c558000 con 0x55b87c13bc00 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.009+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94250847018880] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.009+0000 7f0d0bf78a80 20 remove_watcher() i=1 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.009+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94250847020032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c558400 con 0x55b87c4e7000 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.010+0000 7f0d09f22700 1 -- 172.21.15.191:0/4051528170 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94250847020032] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561440 con 0x55b87c4e7000 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.010+0000 7f0d0bf78a80 20 remove_watcher() i=2 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.010+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94250847021184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c558800 con 0x55b87c56a000 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.011+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94250847021184] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c561680 con 0x55b87c56a000 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.011+0000 7f0d0bf78a80 20 remove_watcher() i=3 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.011+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94250847022336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c558c00 con 0x55b87c13bc00 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.012+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94250847022336] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c13bc00 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.012+0000 7f0d0bf78a80 20 remove_watcher() i=4 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.012+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94250847023488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c559000 con 0x55b87c4e7400 2024-04-27T04:34:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.013+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94250847023488] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.013+0000 7f0d0bf78a80 20 remove_watcher() i=5 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.013+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94250847024640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c559400 con 0x55b87c4e7400 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.014+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94250847024640] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.014+0000 7f0d0bf78a80 20 remove_watcher() i=6 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.014+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94250847025792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b87c559800 con 0x55b87c4e7400 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0a723700 1 -- 172.21.15.191:0/4051528170 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94250847025792] v31'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b87c560d80 con 0x55b87c4e7400 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 20 remove_watcher() i=7 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e7400 msgr2=0x55b87c425180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e7400 0x55b87c425180 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c13bc00 msgr2=0x55b87c423b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c13bc00 0x55b87c423b80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c56a000 msgr2=0x55b87c424c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c56a000 0x55b87c424c00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c4e7000 msgr2=0x55b87c424100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c4e7000 0x55b87c424100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c4e6c00 msgr2=0x55b87c423600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c4e6c00 0x55b87c423600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55b87c41df80 tx=0x55b87c552000 comp rx=0 tx=0).stop 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 msgr2=0x55b87c422b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4051528170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13b800 0x55b87c422b00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55b87cfc6000 tx=0x55b87c48ae40 comp rx=0 tx=0).stop 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 reap_dead start 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d09721700 1 -- 172.21.15.191:0/4051528170 reap_dead start 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 shutdown_connections 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 >> 172.21.15.191:0/4051528170 conn(0x55b87c13ac00 msgr2=0x55b87c227800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 shutdown_connections 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4051528170 wait complete. 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e6400 msgr2=0x55b87c66c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55b87c4e6400 0x55b87c66c000 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c56b800 msgr2=0x55b87c66db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b87c56b800 0x55b87c66db80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c4e7800 msgr2=0x55b87c66e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55b87c4e7800 0x55b87c66e100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c56ac00 msgr2=0x55b87c66d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b87c56ac00 0x55b87c66d600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c56bc00 msgr2=0x55b87c425700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c56bc00 0x55b87c425700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55b87c4e3740 tx=0x55b87c5532c0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 msgr2=0x55b87c424680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d09f22700 1 -- 172.21.15.191:0/4263162921 reap_dead start 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55b87c57e7e0 tx=0x55b87c552240 comp rx=0 tx=0).stop 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 shutdown_connections 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4263162921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c573c00 0x55b87c424680 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.015+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 >> 172.21.15.191:0/4263162921 conn(0x55b87c573800 msgr2=0x55b87c228000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 shutdown_connections 2024-04-27T04:34:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4263162921 wait complete. 2024-04-27T04:34:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c13b400 msgr2=0x55b87c423080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c13b400 0x55b87c423080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55b87cf0de60 tx=0x55b87c48ab40 comp rx=0 tx=0).stop 2024-04-27T04:34:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 msgr2=0x55b87c422580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55b87cf0df20 tx=0x55b87c22b920 comp rx=0 tx=0).stop 2024-04-27T04:34:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 shutdown_connections 2024-04-27T04:34:49.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b87c13b400 0x55b87c423080 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 --2- 172.21.15.191:0/4206639510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b87c13a800 0x55b87c422580 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 >> 172.21.15.191:0/4206639510 conn(0x55b87c13a400 msgr2=0x55b87c227000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 shutdown_connections 2024-04-27T04:34:49.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.016+0000 7f0d0bf78a80 1 -- 172.21.15.191:0/4206639510 wait complete. 2024-04-27T04:34:49.065 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-04-27T04:34:49.065 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-27T04:34:49.065 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-27T04:34:49.066 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-27T04:34:49.144 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:49.144 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:49.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.143+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.143+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.143+0000 7f0480bbea80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a802694c00 0x55a80297c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.143+0000 7f0480bbea80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a8029bf500 con 0x55a802694800 2024-04-27T04:34:49.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.143+0000 7f0480bbea80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a802736a00 con 0x55a802694c00 2024-04-27T04:34:49.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.143+0000 7f047e367700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a802694c00 0x55a80297c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047eb68700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55a802694800 0x55a802780800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:60570/0 (socket says 172.21.15.191:60570) 2024-04-27T04:34:49.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047e367700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a802694c00 0x55a80297c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48000/0 (socket says 172.21.15.191:48000) 2024-04-27T04:34:49.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047eb68700 1 -- 172.21.15.191:0/3497425221 learned_addr learned my addr 172.21.15.191:0/3497425221 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:34:49.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047e367700 1 -- 172.21.15.191:0/3497425221 >> v1:172.21.15.138:6789/0 conn(0x55a802694800 legacy=0x55a802780800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047e367700 1 -- 172.21.15.191:0/3497425221 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bdd40 con 0x55a802694c00 2024-04-27T04:34:49.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047e367700 1 --2- 172.21.15.191:0/3497425221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694c00 0x55a80297c580 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55a802977890 tx=0x55a802785920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e26ee9b038466b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047db66700 1 -- 172.21.15.191:0/3497425221 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d4a0 con 0x55a802694c00 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047db66700 1 -- 172.21.15.191:0/3497425221 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8029bf800 con 0x55a802694c00 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.144+0000 7f047db66700 1 -- 172.21.15.191:0/3497425221 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d600 con 0x55a802694c00 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- 172.21.15.191:0/3497425221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694c00 msgr2=0x55a80297c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 --2- 172.21.15.191:0/3497425221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694c00 0x55a80297c580 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55a802977890 tx=0x55a802785920 comp rx=0 tx=0).stop 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- 172.21.15.191:0/3497425221 shutdown_connections 2024-04-27T04:34:49.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 --2- 172.21.15.191:0/3497425221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694c00 0x55a80297c580 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- 172.21.15.191:0/3497425221 >> 172.21.15.191:0/3497425221 conn(0x55a802694400 msgr2=0x55a802780000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- 172.21.15.191:0/3497425221 shutdown_connections 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- 172.21.15.191:0/3497425221 wait complete. 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.175 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.145+0000 7f0480bbea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a802736500 con 0x55a802694800 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047eb68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047eb68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48016/0 (socket says 172.21.15.191:48016) 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047eb68700 1 -- 172.21.15.191:0/1454926701 learned_addr learned my addr 172.21.15.191:0/1454926701 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047eb68700 1 -- 172.21.15.191:0/1454926701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bdba0 con 0x55a802694800 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047eb68700 1 --2- 172.21.15.191:0/1454926701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55a80346d800 tx=0x55a802785b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c70b7d606a95d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047d365700 1 -- 172.21.15.191:0/1454926701 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d4a0 con 0x55a802694800 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047d365700 1 -- 172.21.15.191:0/1454926701 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8029bfb00 con 0x55a802694800 2024-04-27T04:34:49.176 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f047d365700 1 -- 172.21.15.191:0/1454926701 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d600 con 0x55a802694800 2024-04-27T04:34:49.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 -- 172.21.15.191:0/1454926701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 msgr2=0x55a80297c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1454926701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55a80346d800 tx=0x55a802785b60 comp rx=0 tx=0).stop 2024-04-27T04:34:49.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 -- 172.21.15.191:0/1454926701 shutdown_connections 2024-04-27T04:34:49.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1454926701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 -- 172.21.15.191:0/1454926701 >> 172.21.15.191:0/1454926701 conn(0x55a802694400 msgr2=0x55a802781000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.177 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 -- 172.21.15.191:0/1454926701 shutdown_connections 2024-04-27T04:34:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.146+0000 7f0480bbea80 1 -- 172.21.15.191:0/1454926701 wait complete. 2024-04-27T04:34:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f0480bbea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f0480bbea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8027363c0 con 0x55a802694800 2024-04-27T04:34:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f047eb68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f047eb68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48020/0 (socket says 172.21.15.191:48020) 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f047eb68700 1 -- 172.21.15.191:0/2363922652 learned_addr learned my addr 172.21.15.191:0/2363922652 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f047eb68700 1 -- 172.21.15.191:0/2363922652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bda00 con 0x55a802694800 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f047eb68700 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55a80346da10 tx=0x55a802785a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.147+0000 7f047bb62700 1 -- 172.21.15.191:0/2363922652 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d8c0 con 0x55a802694800 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f047bb62700 1 -- 172.21.15.191:0/2363922652 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a802735c80 con 0x55a802694800 2024-04-27T04:34:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8026bd860 con 0x55a802694800 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f047bb62700 1 -- 172.21.15.191:0/2363922652 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d4a0 con 0x55a802694800 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8026bd6c0 con 0x55a802694800 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f0480bbea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f047bb62700 1 -- 172.21.15.191:0/2363922652 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a803436b00 con 0x55a802694800 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f0480bbea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8026d0640 con 0x55a802695400 2024-04-27T04:34:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f047bb62700 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28000 0x55a80297d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f047bb62700 1 -- 172.21.15.191:0/2363922652 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a80270f180 con 0x55a802694800 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.148+0000 7f047f369700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047f369700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48026/0 (socket says 172.21.15.191:48026) 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047f369700 1 -- 172.21.15.191:0/1186579616 learned_addr learned my addr 172.21.15.191:0/1186579616 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047eb68700 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28000 0x55a80297d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047f369700 1 -- 172.21.15.191:0/1186579616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bc000 con 0x55a802695400 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047f369700 1 --2- 172.21.15.191:0/1186579616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55a80346df80 tx=0x55a8029e6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c734ecd42a804e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047eb68700 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28000 0x55a80297d080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55a803526f90 tx=0x55a8029e6240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047935d700 1 -- 172.21.15.191:0/1186579616 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d8c0 con 0x55a802695400 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047935d700 1 -- 172.21.15.191:0/1186579616 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8029be300 con 0x55a802695400 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f047935d700 1 -- 172.21.15.191:0/1186579616 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d4a0 con 0x55a802695400 2024-04-27T04:34:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 -- 172.21.15.191:0/1186579616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 msgr2=0x55a80297cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1186579616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55a80346df80 tx=0x55a8029e6de0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 -- 172.21.15.191:0/1186579616 shutdown_connections 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1186579616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 -- 172.21.15.191:0/1186579616 >> 172.21.15.191:0/1186579616 conn(0x55a802694c00 msgr2=0x55a802781800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 -- 172.21.15.191:0/1186579616 shutdown_connections 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.149+0000 7f0480bbea80 1 -- 172.21.15.191:0/1186579616 wait complete. 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f0480bbea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f0480bbea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8026d08c0 con 0x55a802695400 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f047f369700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f047f369700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48038/0 (socket says 172.21.15.191:48038) 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 learned_addr learned my addr 172.21.15.191:0/2651862321 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bcea0 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.150+0000 7f047f369700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55a8035268d0 tx=0x55a802785f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.152+0000 7f0477b5a700 1 -- 172.21.15.191:0/2651862321 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274db80 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.152+0000 7f0477b5a700 1 -- 172.21.15.191:0/2651862321 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a802735c80 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.152+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8026bc820 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.152+0000 7f0477b5a700 1 -- 172.21.15.191:0/2651862321 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d8c0 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.152+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8026bc9c0 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f0477b5a700 1 -- 172.21.15.191:0/2651862321 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a803436dc0 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f0477b5a700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28c00 0x55a80297d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f0477b5a700 1 -- 172.21.15.191:0/2651862321 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a80270f340 con 0x55a802695400 2024-04-27T04:34:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f047e367700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28c00 0x55a80297d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a802695800 0x55a80297db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802695c00 con 0x55a802695800 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f047f369700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a802695800 0x55a80297db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f047e367700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28c00 0x55a80297d600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55a802977890 tx=0x55a802785800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f047f369700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a802695800 0x55a80297db80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.153+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 realm 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29000 con 0x55a802695800 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29400 con 0x55a802695800 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29800 con 0x55a802695800 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a802a29c00 0x55a80297e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29000 con 0x55a802a29c00 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.154+0000 7f047eb68700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a802a29c00 0x55a80297e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f047eb68700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a802a29c00 0x55a80297e100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f0480bbea80 20 searching for the correct realm 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29400 con 0x55a802a29c00 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f047eb68700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a802a29400 0x55a80297ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29000 con 0x55a802a29400 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.155+0000 7f047e367700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a802a29400 0x55a80297ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.156+0000 7f047e367700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a802a29400 0x55a80297ec00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.156+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.156+0000 7f047e367700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a802a29000 0x55a80297f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.156+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.156+0000 7f047f369700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a802a29000 0x55a80297f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.156+0000 7f047f369700 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a802a29000 0x55a80297f180 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.157+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29400 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.158+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.158+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.158+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.158+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.158+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.158+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29400 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.159+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.160+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29400 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.161+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.162+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29400 2024-04-27T04:34:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29c00 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29400 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.163+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802695c00 con 0x55a802695800 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea400 con 0x55a802695800 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea800 con 0x55a802a29c00 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 zone default found 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 4 Realm: () 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:49.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 zonegroup default 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeac00 con 0x55a802a29000 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.164+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb000 con 0x55a802a29c00 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb400 con 0x55a802a29c00 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 20 started sync module instance, tier type = 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.165+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb800 con 0x55a802a29400 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.166+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.166+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94180086332160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea800 con 0x55a802a29400 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.166+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb000 con 0x55a802695800 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.167+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94180086332160] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.167+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.167+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94180086333312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb400 con 0x55a802695800 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.167+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea400 con 0x55a802a29c00 2024-04-27T04:34:49.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.168+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94180086333312] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.169+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.169+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94180086334464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeac00 con 0x55a802a29c00 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.169+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802695c00 con 0x55a802a29400 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.170+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94180086334464] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.170+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.170+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94180086335616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802a29800 con 0x55a802a29400 2024-04-27T04:34:49.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.170+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802695800 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.171+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.171+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94180086336768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb800 con 0x55a802695800 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.171+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea800 con 0x55a802a29000 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.171+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94180086335616] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.171+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.171+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94180086337920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aebc00 con 0x55a802a29000 2024-04-27T04:34:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.172+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb000 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.172+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94180086336768] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.173+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.173+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94180086337920] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.173+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94180086339072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb400 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.173+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea000 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94180086339072] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94180086340224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb800 con 0x55a802a29000 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=0 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=1 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=2 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=3 2024-04-27T04:34:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=4 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=5 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.174+0000 7f0480bbea80 20 add_watcher() i=6 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.175+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94180086340224] v31'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.175+0000 7f0480bbea80 20 add_watcher() i=7 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.175+0000 7f0480bbea80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.175+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea800 con 0x55a802695800 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.175+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.175+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94180087922688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aebc00 con 0x55a802695800 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94180087922688] v31'154 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0465b36700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0480bbea80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0480bbea80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0480bbea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.176+0000 7f0480bbea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8027363c0 con 0x55a802aeb400 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f047e367700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f047e367700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48054/0 (socket says 172.21.15.191:48054) 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f047e367700 1 -- 172.21.15.191:0/2977246585 learned_addr learned my addr 172.21.15.191:0/2977246585 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f047e367700 1 -- 172.21.15.191:0/2977246585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bcb60 con 0x55a802aeb400 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f047e367700 1 --2- 172.21.15.191:0/2977246585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55a802b00060 tx=0x55a802785980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be87c1548b39f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f0465335700 1 -- 172.21.15.191:0/2977246585 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a80274d760 con 0x55a802aeb400 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f0465335700 1 -- 172.21.15.191:0/2977246585 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a802b02000 con 0x55a802aeb400 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.177+0000 7f0465335700 1 -- 172.21.15.191:0/2977246585 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a802b0e000 con 0x55a802aeb400 2024-04-27T04:34:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- 172.21.15.191:0/2977246585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 msgr2=0x55a80297e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2977246585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55a802b00060 tx=0x55a802785980 comp rx=0 tx=0).stop 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- 172.21.15.191:0/2977246585 shutdown_connections 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2977246585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- 172.21.15.191:0/2977246585 >> 172.21.15.191:0/2977246585 conn(0x55a802aeb000 msgr2=0x55a802782000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- 172.21.15.191:0/2977246585 shutdown_connections 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- 172.21.15.191:0/2977246585 wait complete. 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 Processor -- start 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- start start 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.178+0000 7f0480bbea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a802736500 con 0x55a802aeb400 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f047e367700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f047e367700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48070/0 (socket says 172.21.15.191:48070) 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f047e367700 1 -- 172.21.15.191:0/1196007005 learned_addr learned my addr 172.21.15.191:0/1196007005 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f047e367700 1 -- 172.21.15.191:0/1196007005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8026bcd00 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f047e367700 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55a802b00270 tx=0x55a8027858c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f0463b32700 1 -- 172.21.15.191:0/1196007005 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a802b0e160 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f0463b32700 1 -- 172.21.15.191:0/1196007005 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a802b02180 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8026bc340 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f0463b32700 1 -- 172.21.15.191:0/1196007005 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a802b0e2c0 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.179+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8026bdd40 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.180+0000 7f0463b32700 1 -- 172.21.15.191:0/1196007005 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a803437b80 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.180+0000 7f0463b32700 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802b8a000 0x55a80297f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.180+0000 7f0463b32700 1 -- 172.21.15.191:0/1196007005 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a80270f500 con 0x55a802aeb400 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.180+0000 7f047f369700 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802b8a000 0x55a80297f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.180+0000 7f0480bbea80 5 note: GC not initialized 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.180+0000 7f047f369700 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802b8a000 0x55a80297f700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55a802a8db30 tx=0x55a802b0d260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f046132d700 20 reqs_thread_entry: start 2024-04-27T04:34:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f0480bbea80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f0480bbea80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f045f329700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:19 2024 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f045eb07700 20 reqs_thread_entry: start 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f0480bbea80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.181+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb800 con 0x55a802695800 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea800 con 0x55a802a29400 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv27 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 20 disabling bucket name=myfoo 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8a400 con 0x55a802a29400 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.182+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8a800 con 0x55a802a29000 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(65 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.183+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8b000 con 0x55a802a29000 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(66 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v31'6 uv6 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 10 moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [notify cookie 94180087924992 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8b800 con 0x55a802a29400 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify (1) cookie 94180086335616 notify 133143986195 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55a802b55ba0 con 0x55a802a29400 2024-04-27T04:34:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047935d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986195 cookie 94180086335616 notifier 4636 bl.length()=725 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(67 notify.3 [notify cookie 94180087924992 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047935d700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047935d700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047935d700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047935d700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aea800 con 0x55a802a29400 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(68 notify.3 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== watch-notify(notify_complete (2) cookie 94180087924992 notify 133143986195 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a802b55d40 con 0x55a802a29400 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_sources= new_sources= 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential sources added=[] removed=[] 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_dests= new_dests= 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential dests added=[] removed=[] 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.185+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8a400 con 0x55a802695800 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.186+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v31'12 uv12 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.186+0000 7f0480bbea80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.186+0000 7f0480bbea80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.186+0000 7f0480bbea80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.186+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94180087924992 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8b800 con 0x55a802695800 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94180086336768 notify 133143986190 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55a802a8fd40 con 0x55a802695800 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047935d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986190 cookie 94180086336768 notifier 4636 bl.length()=533 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(70 notify.4 [notify cookie 94180087924992 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047935d700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047935d700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047935d700 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802695c00 con 0x55a802695800 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.187+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94180087924992 notify 133143986190 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a802b53a00 con 0x55a802695800 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 1, 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:49.219 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:49.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:49.221 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.188+0000 7f0465b36700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.188+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94180087922688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8a800 con 0x55a802695800 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.189+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94180087922688] v31'155 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.189+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94180086332160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8b400 con 0x55a802a29400 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.190+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94180086332160] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.190+0000 7f0480bbea80 20 remove_watcher() i=0 2024-04-27T04:34:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.190+0000 7f0480bbea80 2 removed watcher, disabling cache 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.190+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94180086333312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8ac00 con 0x55a802695800 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.191+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94180086333312] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.191+0000 7f0480bbea80 20 remove_watcher() i=1 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.191+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94180086334464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8b000 con 0x55a802a29c00 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.192+0000 7f047eb68700 1 -- 172.21.15.191:0/2651862321 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94180086334464] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb200 con 0x55a802a29c00 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.192+0000 7f0480bbea80 20 remove_watcher() i=2 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.192+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94180086335616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802b8a400 con 0x55a802a29400 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.193+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94180086335616] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adb440 con 0x55a802a29400 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.193+0000 7f0480bbea80 20 remove_watcher() i=3 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.193+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94180086336768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aebc00 con 0x55a802695800 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.194+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94180086336768] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802695800 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.194+0000 7f0480bbea80 20 remove_watcher() i=4 2024-04-27T04:34:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.194+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94180086337920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802aeb800 con 0x55a802a29000 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.195+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94180086337920] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.195+0000 7f0480bbea80 20 remove_watcher() i=5 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.195+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94180086339072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802ac6000 con 0x55a802a29000 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.196+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94180086339072] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.196+0000 7f0480bbea80 20 remove_watcher() i=6 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.196+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94180086340224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a802ac6400 con 0x55a802a29000 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f047f369700 1 -- 172.21.15.191:0/2651862321 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94180086340224] v31'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a802adab40 con 0x55a802a29000 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 20 remove_watcher() i=7 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a802a29000 msgr2=0x55a80297f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a802a29000 0x55a80297f180 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a802695800 msgr2=0x55a80297db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a802695800 0x55a80297db80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a802a29400 msgr2=0x55a80297ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a802a29400 0x55a80297ec00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a802a29c00 msgr2=0x55a80297e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a802a29c00 0x55a80297e100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28c00 msgr2=0x55a80297d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28c00 0x55a80297d600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55a802977890 tx=0x55a802785800 comp rx=0 tx=0).stop 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 msgr2=0x55a80297cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2651862321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802695400 0x55a80297cb00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55a8035268d0 tx=0x55a802785f20 comp rx=0 tx=0).stop 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 reap_dead start 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f047e367700 1 -- 172.21.15.191:0/2651862321 reap_dead start 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 shutdown_connections 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 >> 172.21.15.191:0/2651862321 conn(0x55a802694c00 msgr2=0x55a802781800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 shutdown_connections 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.197+0000 7f0480bbea80 1 -- 172.21.15.191:0/2651862321 wait complete. 2024-04-27T04:34:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802b8a000 msgr2=0x55a80297f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802b8a000 0x55a80297f700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55a802a8db30 tx=0x55a802b0d260 comp rx=0 tx=0).stop 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 msgr2=0x55a80297e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55a802b00270 tx=0x55a8027858c0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 shutdown_connections 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802b8a000 0x55a80297f700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/1196007005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802aeb400 0x55a80297e680 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 >> 172.21.15.191:0/1196007005 conn(0x55a802aeb000 msgr2=0x55a802782000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 shutdown_connections 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/1196007005 wait complete. 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28000 msgr2=0x55a80297d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28000 0x55a80297d080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55a803526f90 tx=0x55a8029e6240 comp rx=0 tx=0).stop 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 msgr2=0x55a80297c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55a80346da10 tx=0x55a802785a40 comp rx=0 tx=0).stop 2024-04-27T04:34:49.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 shutdown_connections 2024-04-27T04:34:49.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a802a28000 0x55a80297d080 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 --2- 172.21.15.191:0/2363922652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a802694800 0x55a80297c580 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 >> 172.21.15.191:0/2363922652 conn(0x55a802694400 msgr2=0x55a802781000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 shutdown_connections 2024-04-27T04:34:49.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.198+0000 7f0480bbea80 1 -- 172.21.15.191:0/2363922652 wait complete. 2024-04-27T04:34:49.232 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:49.234 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-27T04:34:49.234 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-27T04:34:49.234 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-27T04:34:49.280 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:34:49.280 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:34:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.279+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.279+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.279+0000 7f90f7846a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.279+0000 7f90f7846a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a0be1c1500 con 0x55a0bde96c00 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.279+0000 7f90f7846a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a0bdf38a00 con 0x55a0bde96800 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.279+0000 7f90f57f0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f57f0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48072/0 (socket says 172.21.15.191:48072) 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f57f0700 1 -- 172.21.15.191:0/1529140472 learned_addr learned my addr 172.21.15.191:0/1529140472 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f57f0700 1 -- 172.21.15.191:0/1529140472 >> v1:172.21.15.138:6789/0 conn(0x55a0bde96c00 legacy=0x55a0bdf82800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f57f0700 1 -- 172.21.15.191:0/1529140472 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebfd40 con 0x55a0bde96800 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f57f0700 1 --2- 172.21.15.191:0/1529140472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55a0be1799e0 tx=0x55a0bdf87920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3184dbfbccfa40f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f47ee700 1 -- 172.21.15.191:0/1529140472 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f4a0 con 0x55a0bde96800 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f47ee700 1 -- 172.21.15.191:0/1529140472 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0be1c1800 con 0x55a0bde96800 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.280+0000 7f90f47ee700 1 -- 172.21.15.191:0/1529140472 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f600 con 0x55a0bde96800 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 -- 172.21.15.191:0/1529140472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 msgr2=0x55a0be17e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 --2- 172.21.15.191:0/1529140472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55a0be1799e0 tx=0x55a0bdf87920 comp rx=0 tx=0).stop 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 -- 172.21.15.191:0/1529140472 shutdown_connections 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 --2- 172.21.15.191:0/1529140472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 -- 172.21.15.191:0/1529140472 >> 172.21.15.191:0/1529140472 conn(0x55a0bde96400 msgr2=0x55a0bdf82000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 -- 172.21.15.191:0/1529140472 shutdown_connections 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 -- 172.21.15.191:0/1529140472 wait complete. 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.281+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f7846a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f7846a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0bdf38500 con 0x55a0bde96800 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f57f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f57f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48082/0 (socket says 172.21.15.191:48082) 2024-04-27T04:34:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f57f0700 1 -- 172.21.15.191:0/1059553945 learned_addr learned my addr 172.21.15.191:0/1059553945 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f57f0700 1 -- 172.21.15.191:0/1059553945 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebfba0 con 0x55a0bde96800 2024-04-27T04:34:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f57f0700 1 --2- 172.21.15.191:0/1059553945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a0be178b70 tx=0x55a0bdf87800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc31781236c8294 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f3fed700 1 -- 172.21.15.191:0/1059553945 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f760 con 0x55a0bde96800 2024-04-27T04:34:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f3fed700 1 -- 172.21.15.191:0/1059553945 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0be1c1b00 con 0x55a0bde96800 2024-04-27T04:34:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.282+0000 7f90f3fed700 1 -- 172.21.15.191:0/1059553945 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f1e0 con 0x55a0bde96800 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- 172.21.15.191:0/1059553945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 msgr2=0x55a0be17e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 --2- 172.21.15.191:0/1059553945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a0be178b70 tx=0x55a0bdf87800 comp rx=0 tx=0).stop 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- 172.21.15.191:0/1059553945 shutdown_connections 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 --2- 172.21.15.191:0/1059553945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- 172.21.15.191:0/1059553945 >> 172.21.15.191:0/1059553945 conn(0x55a0bde96400 msgr2=0x55a0bdf83000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- 172.21.15.191:0/1059553945 shutdown_connections 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- 172.21.15.191:0/1059553945 wait complete. 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f7846a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0bdf383c0 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f57f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f57f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48088/0 (socket says 172.21.15.191:48088) 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.283+0000 7f90f57f0700 1 -- 172.21.15.191:0/3227623483 learned_addr learned my addr 172.21.15.191:0/3227623483 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f57f0700 1 -- 172.21.15.191:0/3227623483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebfa00 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f57f0700 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55a0bec5d890 tx=0x55a0bdf87920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f27ea700 1 -- 172.21.15.191:0/3227623483 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f4a0 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f27ea700 1 -- 172.21.15.191:0/3227623483 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0be1c1c80 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f27ea700 1 -- 172.21.15.191:0/3227623483 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f600 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a0bdebf860 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.284+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0bdebf6c0 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f27ea700 1 -- 172.21.15.191:0/3227623483 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a0bec38b00 con 0x55a0bde96800 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f7846a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f7846a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0bded2640 con 0x55a0bde97400 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f27ea700 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21e000 0x55a0be17f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f27ea700 1 -- 172.21.15.191:0/3227623483 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a0bdf11180 con 0x55a0bde96800 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f5ff1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f57f0700 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21e000 0x55a0be17f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f5ff1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48098/0 (socket says 172.21.15.191:48098) 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f5ff1700 1 -- 172.21.15.191:0/3188609087 learned_addr learned my addr 172.21.15.191:0/3188609087 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f5ff1700 1 -- 172.21.15.191:0/3188609087 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebe000 con 0x55a0bde97400 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f57f0700 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21e000 0x55a0be17f080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55a0bed1e9f0 tx=0x55a0be1e4b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/3188609087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55a0bed1e5a0 tx=0x55a0be1e4e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d272699a611ffe84 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90effe5700 1 -- 172.21.15.191:0/3188609087 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f4a0 con 0x55a0bde97400 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90effe5700 1 -- 172.21.15.191:0/3188609087 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0bdf37c80 con 0x55a0bde97400 2024-04-27T04:34:49.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.285+0000 7f90effe5700 1 -- 172.21.15.191:0/3188609087 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f600 con 0x55a0bde97400 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- 172.21.15.191:0/3188609087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 msgr2=0x55a0be17eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3188609087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55a0bed1e5a0 tx=0x55a0be1e4e40 comp rx=0 tx=0).stop 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- 172.21.15.191:0/3188609087 shutdown_connections 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3188609087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- 172.21.15.191:0/3188609087 >> 172.21.15.191:0/3188609087 conn(0x55a0bde96c00 msgr2=0x55a0bdf83800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- 172.21.15.191:0/3188609087 shutdown_connections 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- 172.21.15.191:0/3188609087 wait complete. 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f7846a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0bded28c0 con 0x55a0bde97400 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.286+0000 7f90f5ff1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90f5ff1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48114/0 (socket says 172.21.15.191:48114) 2024-04-27T04:34:49.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 learned_addr learned my addr 172.21.15.191:0/2255434987 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebeea0 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55a0bed1fe90 tx=0x55a0be1e4a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90ee7e2700 1 -- 172.21.15.191:0/2255434987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f760 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90ee7e2700 1 -- 172.21.15.191:0/2255434987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0bdf37b00 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90ee7e2700 1 -- 172.21.15.191:0/2255434987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0bdf4f4a0 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a0bdebe820 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.287+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0bdebe9c0 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90ee7e2700 1 -- 172.21.15.191:0/2255434987 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a0bec38dc0 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90ee7e2700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21ec00 0x55a0be17f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90ee7e2700 1 -- 172.21.15.191:0/2255434987 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a0bdf11340 con 0x55a0bde97400 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f4fef700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21ec00 0x55a0be17f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a0bde97800 0x55a0be17fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0bde97c00 con 0x55a0bde97800 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a0bde97800 0x55a0be17fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f4fef700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21ec00 0x55a0be17f600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a0be179f80 tx=0x55a0be2d40c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.288+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a0bde97800 0x55a0be17fb80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 realm 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f000 con 0x55a0bde97800 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f400 con 0x55a0bde97800 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f800 con 0x55a0bde97800 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a0be21fc00 0x55a0be180100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f000 con 0x55a0be21fc00 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.289+0000 7f90f57f0700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a0be21fc00 0x55a0be180100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f57f0700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a0be21fc00 0x55a0be180100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f7846a80 20 searching for the correct realm 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f400 con 0x55a0be21fc00 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f57f0700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a0be21f400 0x55a0be180c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f000 con 0x55a0be21f400 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.290+0000 7f90f4fef700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a0be21f400 0x55a0be180c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.291+0000 7f90f4fef700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a0be21f400 0x55a0be180c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.291+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.291+0000 7f90f4fef700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a0be21f000 0x55a0be181180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.291+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.291+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a0be21f000 0x55a0be181180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.291+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a0be21f000 0x55a0be181180 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.292+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f400 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.293+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f400 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.294+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.295+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f400 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.296+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f400 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.297+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21fc00 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f400 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f7846a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f7846a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f7846a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f7846a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.298+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0bde97c00 con 0x55a0bde97800 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8400 con 0x55a0bde97800 2024-04-27T04:34:49.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8800 con 0x55a0be21fc00 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 zone default found 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 4 Realm: () 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 zonegroup default 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8c00 con 0x55a0be21f000 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.299+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9000 con 0x55a0be21fc00 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9400 con 0x55a0be21fc00 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 20 started sync module instance, tier type = 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.300+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9800 con 0x55a0be21f400 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.301+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.301+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94148872329984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8800 con 0x55a0be21f400 2024-04-27T04:34:49.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.301+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9000 con 0x55a0bde97800 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.302+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94148872329984] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.302+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.302+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94148872331136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9400 con 0x55a0bde97800 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.302+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8400 con 0x55a0be21fc00 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.303+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.303+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94148872332288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8c00 con 0x55a0be21fc00 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.303+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0bde97c00 con 0x55a0be21f400 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.303+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94148872331136] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.304+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94148872332288] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.304+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.304+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94148872333440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be21f800 con 0x55a0be21f400 2024-04-27T04:34:49.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.304+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0bde97800 2024-04-27T04:34:49.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.305+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.305+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94148872334592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9800 con 0x55a0bde97800 2024-04-27T04:34:49.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.305+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8800 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.305+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94148872333440] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.306+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.306+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94148872335744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9c00 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.306+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9000 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.306+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94148872334592] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.307+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94148872335744] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.307+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.307+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94148872336896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9400 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.307+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8000 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94148872336896] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94148872338048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9800 con 0x55a0be21f000 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=0 2024-04-27T04:34:49.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=1 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=2 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=3 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=4 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=5 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.309+0000 7f90f7846a80 20 add_watcher() i=6 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.310+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94148872338048] v31'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.310+0000 7f90f7846a80 20 add_watcher() i=7 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.310+0000 7f90f7846a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.310+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8800 con 0x55a0bde97800 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.310+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.310+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94148873895936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9c00 con 0x55a0bde97800 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.311+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94148873895936] v31'156 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.311+0000 7f90dc7be700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.311+0000 7f90f7846a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.311+0000 7f90f7846a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.311+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f7846a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f7846a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0bdf383c0 con 0x55a0be2e9400 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f4fef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f4fef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48130/0 (socket says 172.21.15.191:48130) 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f4fef700 1 -- 172.21.15.191:0/260290762 learned_addr learned my addr 172.21.15.191:0/260290762 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f4fef700 1 -- 172.21.15.191:0/260290762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebeb60 con 0x55a0be2e9400 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90f4fef700 1 --2- 172.21.15.191:0/260290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55a0be22f410 tx=0x55a0be1e5f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44d6160b73695c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90dbfbd700 1 -- 172.21.15.191:0/260290762 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0be308000 con 0x55a0be2e9400 2024-04-27T04:34:49.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90dbfbd700 1 -- 172.21.15.191:0/260290762 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0be30a000 con 0x55a0be2e9400 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.312+0000 7f90dbfbd700 1 -- 172.21.15.191:0/260290762 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0be308160 con 0x55a0be2e9400 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 -- 172.21.15.191:0/260290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 msgr2=0x55a0be180680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 --2- 172.21.15.191:0/260290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55a0be22f410 tx=0x55a0be1e5f80 comp rx=0 tx=0).stop 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 -- 172.21.15.191:0/260290762 shutdown_connections 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 --2- 172.21.15.191:0/260290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 -- 172.21.15.191:0/260290762 >> 172.21.15.191:0/260290762 conn(0x55a0be2e9000 msgr2=0x55a0bdf84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 -- 172.21.15.191:0/260290762 shutdown_connections 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 -- 172.21.15.191:0/260290762 wait complete. 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 Processor -- start 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.313+0000 7f90f7846a80 1 -- start start 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f7846a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f7846a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0bdf38500 con 0x55a0be2e9400 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f4fef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f4fef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:48136/0 (socket says 172.21.15.191:48136) 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f4fef700 1 -- 172.21.15.191:0/3236099389 learned_addr learned my addr 172.21.15.191:0/3236099389 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f4fef700 1 -- 172.21.15.191:0/3236099389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0bdebed00 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f4fef700 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55a0be300180 tx=0x55a0be2d44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90da7ba700 1 -- 172.21.15.191:0/3236099389 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0be3082c0 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90da7ba700 1 -- 172.21.15.191:0/3236099389 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0be30a180 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a0bdebe340 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90da7ba700 1 -- 172.21.15.191:0/3236099389 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a0be308000 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.314+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0bdebeb60 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.315+0000 7f90da7ba700 1 -- 172.21.15.191:0/3236099389 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a0bec39b80 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.315+0000 7f90da7ba700 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be38a000 0x55a0be181700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.315+0000 7f90da7ba700 1 -- 172.21.15.191:0/3236099389 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a0bdf11500 con 0x55a0be2e9400 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.315+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be38a000 0x55a0be181700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.315+0000 7f90f7846a80 5 note: GC not initialized 2024-04-27T04:34:49.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90f5ff1700 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be38a000 0x55a0be181700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55a0be282bd0 tx=0x55a0be2d4360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90d7fb5700 20 reqs_thread_entry: start 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90f7846a80 20 init_complete bucket index max shards: 11 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90f7846a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90d5fb1700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:19 2024 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90d578f700 20 reqs_thread_entry: start 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90f7846a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.316+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9800 con 0x55a0bde97800 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv12 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:49.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8800 con 0x55a0be21f400 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv27 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 20 enabling bucket name=myfoo 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.317+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38a400 con 0x55a0be21f400 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 : miss 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 20 rados->read ofs=0 len=0 2024-04-27T04:34:49.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38a800 con 0x55a0be21f000 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(65 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv6 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.318+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 8.1e 8:789222e7:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38b000 con 0x55a0be21f000 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(66 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v31'7 uv7 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 10 moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1, ofs0, ns] 2024-04-27T04:34:49.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [notify cookie 94148873898240 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38b800 con 0x55a0be21f400 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== watch-notify(notify (1) cookie 94148872333440 notify 133143986196 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55a0be285ba0 con 0x55a0be21f400 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90effe5700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986196 cookie 94148872333440 notifier 4644 bl.length()=725 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(67 notify.3 [notify cookie 94148873898240 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90effe5700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 info.flags=0x17 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90effe5700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1 to cache LRU end 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90effe5700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90effe5700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e8800 con 0x55a0be21f400 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(68 notify.3 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== watch-notify(notify_complete (2) cookie 94148873898240 notify 133143986196 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a0be2841a0 con 0x55a0be21f400 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_sources= new_sources= 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential sources added=[] removed=[] 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): orig_dests= new_dests= 2024-04-27T04:34:49.341 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 20 handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1]): potential dests added=[] removed=[] 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.320+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38a400 con 0x55a0bde97800 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v31'13 uv13 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f7846a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f7846a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f7846a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94148873898240 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38b800 con 0x55a0bde97800 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94148872334592 notify 133143986191 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55a0be285d40 con 0x55a0bde97800 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90effe5700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 94148872334592 notifier 4644 bl.length()=533 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(70 notify.4 [notify cookie 94148873898240 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90effe5700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90effe5700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90effe5700 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0bde97c00 con 0x55a0bde97800 2024-04-27T04:34:49.342 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.322+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94148873898240 notify 133143986191 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a0be355d40 con 0x55a0bde97800 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:34:49.343 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:34:49.344 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:34:49.345 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:34:49.346 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:34:49.346 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:34:49.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.323+0000 7f90dc7be700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:34:49.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.323+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94148873895936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38a800 con 0x55a0bde97800 2024-04-27T04:34:49.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.324+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94148873895936] v31'157 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.324+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94148872329984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38b400 con 0x55a0be21f400 2024-04-27T04:34:49.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.325+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94148872329984] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.325+0000 7f90f7846a80 20 remove_watcher() i=0 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.325+0000 7f90f7846a80 2 removed watcher, disabling cache 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.325+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94148872331136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38ac00 con 0x55a0bde97800 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.326+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94148872331136] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.326+0000 7f90f7846a80 20 remove_watcher() i=1 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.326+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94148872332288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38b000 con 0x55a0be21fc00 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.327+0000 7f90f57f0700 1 -- 172.21.15.191:0/2255434987 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94148872332288] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df200 con 0x55a0be21fc00 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.327+0000 7f90f7846a80 20 remove_watcher() i=2 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.327+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94148872333440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be38a400 con 0x55a0be21f400 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.328+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94148872333440] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2df440 con 0x55a0be21f400 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.328+0000 7f90f7846a80 20 remove_watcher() i=3 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.328+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94148872334592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9c00 con 0x55a0bde97800 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.329+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94148872334592] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0bde97800 2024-04-27T04:34:49.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.329+0000 7f90f7846a80 20 remove_watcher() i=4 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.329+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94148872335744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2e9800 con 0x55a0be21f000 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.330+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94148872335744] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.330+0000 7f90f7846a80 20 remove_watcher() i=5 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.330+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94148872336896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2d6000 con 0x55a0be21f000 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.331+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94148872336896] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.331+0000 7f90f7846a80 20 remove_watcher() i=6 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.331+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94148872338048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a0be2d6400 con 0x55a0be21f000 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f5ff1700 1 -- 172.21.15.191:0/2255434987 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94148872338048] v31'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a0be2deb40 con 0x55a0be21f000 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 20 remove_watcher() i=7 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a0be21f000 msgr2=0x55a0be181180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a0be21f000 0x55a0be181180 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a0bde97800 msgr2=0x55a0be17fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a0bde97800 0x55a0be17fb80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a0be21f400 msgr2=0x55a0be180c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a0be21f400 0x55a0be180c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a0be21fc00 msgr2=0x55a0be180100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a0be21fc00 0x55a0be180100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21ec00 msgr2=0x55a0be17f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21ec00 0x55a0be17f600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a0be179f80 tx=0x55a0be2d40c0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 msgr2=0x55a0be17eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 --2- 172.21.15.191:0/2255434987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde97400 0x55a0be17eb00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55a0bed1fe90 tx=0x55a0be1e4a20 comp rx=0 tx=0).stop 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 reap_dead start 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f4fef700 1 -- 172.21.15.191:0/2255434987 reap_dead start 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 shutdown_connections 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 >> 172.21.15.191:0/2255434987 conn(0x55a0bde96c00 msgr2=0x55a0bdf83800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 shutdown_connections 2024-04-27T04:34:49.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.332+0000 7f90f7846a80 1 -- 172.21.15.191:0/2255434987 wait complete. 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be38a000 msgr2=0x55a0be181700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be38a000 0x55a0be181700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55a0be282bd0 tx=0x55a0be2d4360 comp rx=0 tx=0).stop 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 msgr2=0x55a0be180680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55a0be300180 tx=0x55a0be2d44e0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 shutdown_connections 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be38a000 0x55a0be181700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3236099389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0be2e9400 0x55a0be180680 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 >> 172.21.15.191:0/3236099389 conn(0x55a0be2e9000 msgr2=0x55a0bdf84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 shutdown_connections 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3236099389 wait complete. 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21e000 msgr2=0x55a0be17f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21e000 0x55a0be17f080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55a0bed1e9f0 tx=0x55a0be1e4b40 comp rx=0 tx=0).stop 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 msgr2=0x55a0be17e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:34:49.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55a0bec5d890 tx=0x55a0bdf87920 comp rx=0 tx=0).stop 2024-04-27T04:34:49.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 shutdown_connections 2024-04-27T04:34:49.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a0be21e000 0x55a0be17f080 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 --2- 172.21.15.191:0/3227623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0bde96800 0x55a0be17e580 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:34:49.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 >> 172.21.15.191:0/3227623483 conn(0x55a0bde96400 msgr2=0x55a0bdf83000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:34:49.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 shutdown_connections 2024-04-27T04:34:49.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:34:49.333+0000 7f90f7846a80 1 -- 172.21.15.191:0/3227623483 wait complete. 2024-04-27T04:34:49.356 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:34:49.363 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T04:34:49.608 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-04-27T04:34:49.614 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:04.615 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-27T04:35:04.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-27T04:35:04.616 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-27T04:35:04.668 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:04.668 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:04.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.667+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.667+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64c02e0a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64c02e0a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564dc6def500 con 0x564dc6ac4c00 2024-04-27T04:35:04.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64c02e0a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x564dc6b66a00 con 0x564dc6ac4800 2024-04-27T04:35:04.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64be28a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64bda89700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x564dc6ac4c00 0x564dc6bb0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:33536/0 (socket says 172.21.15.191:33536) 2024-04-27T04:35:04.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64bda89700 1 -- 172.21.15.191:0/1418076625 learned_addr learned my addr 172.21.15.191:0/1418076625 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:04.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64be28a700 1 -- 172.21.15.191:0/1418076625 >> v1:172.21.15.138:6789/0 conn(0x564dc6ac4c00 legacy=0x564dc6bb0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64be28a700 1 -- 172.21.15.191:0/1418076625 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aedd40 con 0x564dc6ac4800 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.668+0000 7f64bd288700 1 -- 172.21.15.191:0/1418076625 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127273658 0 0) 0x564dc7892000 con 0x564dc6ac4c00 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64be28a700 1 --2- 172.21.15.191:0/1418076625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x564dc6da79e0 tx=0x564dc6bb5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fe377befe43ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64bd288700 1 -- 172.21.15.191:0/1418076625 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d4a0 con 0x564dc6ac4800 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64bd288700 1 -- 172.21.15.191:0/1418076625 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6def800 con 0x564dc6ac4800 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64bd288700 1 -- 172.21.15.191:0/1418076625 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d600 con 0x564dc6ac4800 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 -- 172.21.15.191:0/1418076625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 msgr2=0x564dc6dac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/1418076625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x564dc6da79e0 tx=0x564dc6bb5920 comp rx=0 tx=0).stop 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 -- 172.21.15.191:0/1418076625 shutdown_connections 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/1418076625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 -- 172.21.15.191:0/1418076625 >> 172.21.15.191:0/1418076625 conn(0x564dc6ac4400 msgr2=0x564dc6bb0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 -- 172.21.15.191:0/1418076625 shutdown_connections 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.669+0000 7f64c02e0a80 1 -- 172.21.15.191:0/1418076625 wait complete. 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.670+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.670+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.670+0000 7f64c02e0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.670+0000 7f64c02e0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6b66500 con 0x564dc6ac4800 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64be28a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64be28a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45882/0 (socket says 172.21.15.191:45882) 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64be28a700 1 -- 172.21.15.191:0/708502989 learned_addr learned my addr 172.21.15.191:0/708502989 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64be28a700 1 -- 172.21.15.191:0/708502989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aedba0 con 0x564dc6ac4800 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64be28a700 1 --2- 172.21.15.191:0/708502989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x564dc7895800 tx=0x564dc6bb5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2becba76d5d97bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64bca87700 1 -- 172.21.15.191:0/708502989 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d760 con 0x564dc6ac4800 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64bca87700 1 -- 172.21.15.191:0/708502989 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6defb00 con 0x564dc6ac4800 2024-04-27T04:35:04.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.671+0000 7f64bca87700 1 -- 172.21.15.191:0/708502989 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d1e0 con 0x564dc6ac4800 2024-04-27T04:35:04.700 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- 172.21.15.191:0/708502989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 msgr2=0x564dc6dac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/708502989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x564dc7895800 tx=0x564dc6bb5800 comp rx=0 tx=0).stop 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- 172.21.15.191:0/708502989 shutdown_connections 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/708502989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- 172.21.15.191:0/708502989 >> 172.21.15.191:0/708502989 conn(0x564dc6ac4400 msgr2=0x564dc6bb1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- 172.21.15.191:0/708502989 shutdown_connections 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- 172.21.15.191:0/708502989 wait complete. 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64c02e0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6b663c0 con 0x564dc6ac4800 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.672+0000 7f64be28a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64be28a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45888/0 (socket says 172.21.15.191:45888) 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64be28a700 1 -- 172.21.15.191:0/2126549615 learned_addr learned my addr 172.21.15.191:0/2126549615 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64be28a700 1 -- 172.21.15.191:0/2126549615 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aeda00 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64be28a700 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x564dc7895ec0 tx=0x564dc6bb5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64bb284700 1 -- 172.21.15.191:0/2126549615 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d4a0 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64bb284700 1 -- 172.21.15.191:0/2126549615 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6defc80 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564dc6aed860 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64bb284700 1 -- 172.21.15.191:0/2126549615 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d600 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.673+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc6aed6c0 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64c02e0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bb284700 1 -- 172.21.15.191:0/2126549615 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564dc7866b00 con 0x564dc6ac4800 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64c02e0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6b66000 con 0x564dc6ac5400 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bb284700 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54000 0x564dc6dad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bb284700 1 -- 172.21.15.191:0/2126549615 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564dc6b3f180 con 0x564dc6ac4800 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bea8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bea8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45902/0 (socket says 172.21.15.191:45902) 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bea8b700 1 -- 172.21.15.191:0/3113305070 learned_addr learned my addr 172.21.15.191:0/3113305070 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64be28a700 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54000 0x564dc6dad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.674+0000 7f64bea8b700 1 -- 172.21.15.191:0/3113305070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aec000 con 0x564dc6ac5400 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64bea8b700 1 --2- 172.21.15.191:0/3113305070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x564dc78959b0 tx=0x564dc6e1ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603a2d1f15e992e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64be28a700 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54000 0x564dc6dad080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x564dc793d050 tx=0x564dc6e1ac60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64b8a7f700 1 -- 172.21.15.191:0/3113305070 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d4a0 con 0x564dc6ac5400 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64b8a7f700 1 -- 172.21.15.191:0/3113305070 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6b65c80 con 0x564dc6ac5400 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64b8a7f700 1 -- 172.21.15.191:0/3113305070 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d600 con 0x564dc6ac5400 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 -- 172.21.15.191:0/3113305070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 msgr2=0x564dc6dacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/3113305070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x564dc78959b0 tx=0x564dc6e1ab40 comp rx=0 tx=0).stop 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 -- 172.21.15.191:0/3113305070 shutdown_connections 2024-04-27T04:35:04.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/3113305070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 -- 172.21.15.191:0/3113305070 >> 172.21.15.191:0/3113305070 conn(0x564dc6ac4c00 msgr2=0x564dc6bb1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 -- 172.21.15.191:0/3113305070 shutdown_connections 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.675+0000 7f64c02e0a80 1 -- 172.21.15.191:0/3113305070 wait complete. 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64c02e0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64c02e0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6b00640 con 0x564dc6ac5400 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64bea8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64bea8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45910/0 (socket says 172.21.15.191:45910) 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 learned_addr learned my addr 172.21.15.191:0/423316132 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aecea0 con 0x564dc6ac5400 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64bea8b700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x564dc793c6c0 tx=0x564dc6bb5ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64b727c700 1 -- 172.21.15.191:0/423316132 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d760 con 0x564dc6ac5400 2024-04-27T04:35:04.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64b727c700 1 -- 172.21.15.191:0/423316132 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6b65b00 con 0x564dc6ac5400 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564dc6aec820 con 0x564dc6ac5400 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.676+0000 7f64b727c700 1 -- 172.21.15.191:0/423316132 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6b7d4a0 con 0x564dc6ac5400 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.677+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc6aec9c0 con 0x564dc6ac5400 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.677+0000 7f64b727c700 1 -- 172.21.15.191:0/423316132 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564dc7866dc0 con 0x564dc6ac5400 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64b727c700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54c00 0x564dc6dad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64b727c700 1 -- 172.21.15.191:0/423316132 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564dc6b3f340 con 0x564dc6ac5400 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64bda89700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54c00 0x564dc6dad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6ac5800 0x564dc6dadb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6ac5800 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64bea8b700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6ac5800 0x564dc6dadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64bda89700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54c00 0x564dc6dad600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x564dc7894030 tx=0x564dc6ef20c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.678+0000 7f64bea8b700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6ac5800 0x564dc6dadb80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 realm 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55000 con 0x564dc6ac5800 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55400 con 0x564dc6ac5800 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55800 con 0x564dc6ac5800 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.679+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6e55c00 0x564dc6dae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55000 con 0x564dc6e55c00 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64be28a700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6e55c00 0x564dc6dae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64be28a700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6e55c00 0x564dc6dae100 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64c02e0a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64c02e0a80 20 searching for the correct realm 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.680+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55400 con 0x564dc6e55c00 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64be28a700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6e55400 0x564dc6daec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55000 con 0x564dc6e55400 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64bda89700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6e55400 0x564dc6daec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64bda89700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6e55400 0x564dc6daec00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64bda89700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6e55000 0x564dc6daf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.681+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.682+0000 7f64bea8b700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6e55000 0x564dc6daf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.682+0000 7f64bea8b700 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6e55000 0x564dc6daf180 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.682+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.682+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.683+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55400 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.684+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.684+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.684+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.684+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.684+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.684+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55400 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.685+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.686+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.687+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.687+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.687+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.687+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.687+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.687+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.688+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.688+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55400 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.688+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.688+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.688+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.711 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.688+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55400 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.689+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55c00 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.690+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.690+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.690+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.690+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.690+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.690+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55400 2024-04-27T04:35:04.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6ac5800 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6ac5800 2024-04-27T04:35:04.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:04.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18800 con 0x564dc6e55c00 2024-04-27T04:35:04.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:04.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 zone default found 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 4 Realm: () 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 zonegroup default 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.691+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6e55000 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19000 con 0x564dc6e55c00 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19400 con 0x564dc6e55c00 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 started sync module instance, tier type = 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.692+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19800 con 0x564dc6e55400 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.693+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.694+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94892048661248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18800 con 0x564dc6e55400 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.694+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19000 con 0x564dc6ac5800 2024-04-27T04:35:04.716 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.695+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94892048661248] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.695+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.695+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94892048662400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19400 con 0x564dc6ac5800 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.695+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6e55c00 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.696+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94892048662400] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.696+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.696+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94892048663552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6e55c00 2024-04-27T04:35:04.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.696+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6e55400 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.698+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.698+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94892048663552] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.698+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94892048664704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6e55800 con 0x564dc6e55400 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.698+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6ac5800 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.699+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94892048664704] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.699+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.699+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94892048665856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19800 con 0x564dc6ac5800 2024-04-27T04:35:04.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.699+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18800 con 0x564dc6e55000 2024-04-27T04:35:04.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.700+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94892048665856] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.700+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.700+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94892048667008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6e55000 2024-04-27T04:35:04.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.700+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19000 con 0x564dc6e55000 2024-04-27T04:35:04.730 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.701+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94892048667008] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.701+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.701+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94892048668160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19400 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.701+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18000 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94892048668160] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(58 notify.7 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94892048669312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19800 con 0x564dc6e55000 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=0 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=1 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=2 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=3 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=4 2024-04-27T04:35:04.731 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=5 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.702+0000 7f64c02e0a80 20 add_watcher() i=6 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.703+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94892048669312] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.703+0000 7f64c02e0a80 20 add_watcher() i=7 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.703+0000 7f64c02e0a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.703+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18800 con 0x564dc6ac5800 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.704+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.704+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94892050235392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6ac5800 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94892050235392] v31'158 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64a5258700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64c02e0a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64c02e0a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.732 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64c02e0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.705+0000 7f64c02e0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6b00780 con 0x564dc6f19400 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64bda89700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64bda89700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45924/0 (socket says 172.21.15.191:45924) 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64bda89700 1 -- 172.21.15.191:0/864877626 learned_addr learned my addr 172.21.15.191:0/864877626 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64bda89700 1 -- 172.21.15.191:0/864877626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aecb60 con 0x564dc6f19400 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64bda89700 1 --2- 172.21.15.191:0/864877626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x564dc6e9d800 tx=0x564dc6e1bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37ff5cc3dc7679 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64a4a57700 1 -- 172.21.15.191:0/864877626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6f36000 con 0x564dc6f19400 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64a4a57700 1 -- 172.21.15.191:0/864877626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6f3a000 con 0x564dc6f19400 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.706+0000 7f64a4a57700 1 -- 172.21.15.191:0/864877626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6f36160 con 0x564dc6f19400 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 -- 172.21.15.191:0/864877626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 msgr2=0x564dc6dae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/864877626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x564dc6e9d800 tx=0x564dc6e1bf80 comp rx=0 tx=0).stop 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 -- 172.21.15.191:0/864877626 shutdown_connections 2024-04-27T04:35:04.733 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/864877626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 -- 172.21.15.191:0/864877626 >> 172.21.15.191:0/864877626 conn(0x564dc6f19000 msgr2=0x564dc6bb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 -- 172.21.15.191:0/864877626 shutdown_connections 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 -- 172.21.15.191:0/864877626 wait complete. 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 Processor -- start 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.707+0000 7f64c02e0a80 1 -- start start 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64c02e0a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.734 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64c02e0a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6b66000 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64bda89700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64bda89700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45938/0 (socket says 172.21.15.191:45938) 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 learned_addr learned my addr 172.21.15.191:0/4265309432 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc6aecd00 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64bda89700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x564dc6f39200 tx=0x564dc6ef24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64a3254700 1 -- 172.21.15.191:0/4265309432 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6f362c0 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64a3254700 1 -- 172.21.15.191:0/4265309432 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc6f3a180 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564dc6aec340 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64a3254700 1 -- 172.21.15.191:0/4265309432 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564dc6f36000 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.708+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc6aecb60 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.709+0000 7f64a3254700 1 -- 172.21.15.191:0/4265309432 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564dc7867b80 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.709+0000 7f64a3254700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6fb8000 0x564dc6daf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.709+0000 7f64a3254700 1 -- 172.21.15.191:0/4265309432 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x564dc6b3f500 con 0x564dc6f19400 2024-04-27T04:35:04.735 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.710+0000 7f64bea8b700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6fb8000 0x564dc6daf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.710+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6f18000 0x564dc6fcc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.710+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19800 con 0x564dc6f18000 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.710+0000 7f64be28a700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6f18000 0x564dc6fcc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.710+0000 7f64bea8b700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6fb8000 0x564dc6daf700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x564dc6e9d470 tx=0x564dc6ef2360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.710+0000 7f64be28a700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6f18000 0x564dc6fcc000 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.711+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'51 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.711+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6f18800 0x564dc6fcc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.711+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8400 con 0x564dc6f18800 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.711+0000 7f64bda89700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6f18800 0x564dc6fcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.712+0000 7f64bda89700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6f18800 0x564dc6fcc580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.713+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'24 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.713+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6fb8800 0x564dc6fccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.736 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.713+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6fb8800 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.713+0000 7f64bea8b700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6fb8800 0x564dc6fccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.713+0000 7f64bea8b700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6fb8800 0x564dc6fccb00 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.714+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.714+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6f18000 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.715+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'159 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.715+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6fb9400 0x564dc6fcd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.715+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6fb9400 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.715+0000 7f64be28a700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6fb9400 0x564dc6fcd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.716+0000 7f64be28a700 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6fb9400 0x564dc6fcd080 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.717+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'53 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.717+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9c00 con 0x564dc6fb8800 2024-04-27T04:35:04.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.718+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'42 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.718+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6f18000 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.719+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.719+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6f18800 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.720+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'37 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.720+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6fb9400 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.721+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.721+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19800 con 0x564dc6fb8800 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.722+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.738 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.722+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6fb9400 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.723+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'54 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.723+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016400 con 0x564dc6f18800 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.725+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'21 uv20 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.725+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6fb9400 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.726+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.726+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016c00 con 0x564dc6f18800 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.727+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'51 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.727+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6f18000 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.728+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'160 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.739 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.728+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017400 con 0x564dc6fb8800 2024-04-27T04:35:04.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.729+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'44 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.729+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017800 con 0x564dc6fb9400 2024-04-27T04:35:04.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.730+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'55 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.730+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6fb8800 2024-04-27T04:35:04.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.731+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'30 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.731+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6f18000 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.732+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'26 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.732+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6f18000 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.733+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'161 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.733+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016400 con 0x564dc6fb8800 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.734+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'45 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.734+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6fb9400 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.735+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'34 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.735+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6fb8800 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.736+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'26 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.736+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6fb8800 2024-04-27T04:35:04.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.737+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'36 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.737+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.738+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.738+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.739+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'43 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.739+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8400 con 0x564dc6f18800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.740+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'26 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.740+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.741+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'30 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.741+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017400 con 0x564dc6f18800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.742+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'38 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.742+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6fb8800 2024-04-27T04:35:04.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.743+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'46 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.743+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016400 con 0x564dc6f18800 2024-04-27T04:35:04.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.745+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'52 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.745+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6fb9400 2024-04-27T04:35:04.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.746+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.747+0000 7f64a0a4f700 20 reqs_thread_entry: start 2024-04-27T04:35:04.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.747+0000 7f64c02e0a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:04.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.748+0000 7f64c02e0a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:04.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.748+0000 7f649ea4b700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:34 2024 2024-04-27T04:35:04.750 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:04.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.748+0000 7f649e229700 20 reqs_thread_entry: start 2024-04-27T04:35:04.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.748+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6f18000 2024-04-27T04:35:04.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.748+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.748+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9c00 con 0x564dc6f18000 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6f18000 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6f18800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv14 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017800 con 0x564dc6f18800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv14 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.749+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6f18800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.750+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv14 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.750+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6fb8800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.750+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.750+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6fb8800 2024-04-27T04:35:04.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.751+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.751+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6fb8800 2024-04-27T04:35:04.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.751+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.751+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6f18000 2024-04-27T04:35:04.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.751+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv19 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.751+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6f18000 2024-04-27T04:35:04.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.752+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv19 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.752+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6f18000 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.752+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv19 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.752+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6fb9400 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6fb9400 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv17 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016c00 con 0x564dc6fb9400 2024-04-27T04:35:04.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017400 con 0x564dc6fb8800 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.753+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.754+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016400 con 0x564dc6fb8800 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.754+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.754+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6fb8800 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.754+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.754+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6f18000 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8400 con 0x564dc6f18000 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6f18000 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6f18800 2024-04-27T04:35:04.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.755+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.756+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6f18800 2024-04-27T04:35:04.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.756+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.756+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6f18800 2024-04-27T04:35:04.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.756+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.756+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6fb9400 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.757+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.757+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6fb9400 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.757+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.757+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9c00 con 0x564dc6fb9400 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.757+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.757+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6fb8800 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.758+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.758+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c000 con 0x564dc6fb8800 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.758+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.758+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c400 con 0x564dc6fb8800 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.759+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.759+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c800 con 0x564dc6fb9400 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.759+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.759+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707cc00 con 0x564dc6fb9400 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d000 con 0x564dc6fb9400 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d400 con 0x564dc6f18800 2024-04-27T04:35:04.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d800 con 0x564dc6f18800 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.760+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707dc00 con 0x564dc6f18800 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.761+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(68 gc.11 [call out=1700b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+1700 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.761+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d400 con 0x564dc6fb9400 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.761+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.761+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c800 con 0x564dc6fb9400 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.761+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.761+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6fb9400 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8400 con 0x564dc6f18800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9c00 con 0x564dc6f18800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6f18800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.762+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6f18000 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6f18000 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017800 con 0x564dc6f18000 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6fb8800 2024-04-27T04:35:04.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.763+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6fb8800 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.764+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.764+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6fb8800 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.764+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.764+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6fb9400 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.764+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.764+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016c00 con 0x564dc6fb9400 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.765+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.765+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017800 con 0x564dc6fb9400 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.765+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.765+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6fb8800 2024-04-27T04:35:04.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.765+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.765+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017400 con 0x564dc6fb8800 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.766+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.766+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016400 con 0x564dc6fb8800 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.766+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.766+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6f18000 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.766+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.766+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6f18000 2024-04-27T04:35:04.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707cc00 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d000 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d400 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.767+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c800 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6f18000 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c000 con 0x564dc6fb8800 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c400 con 0x564dc6fb8800 2024-04-27T04:35:04.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 29 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6fb8800 2024-04-27T04:35:04.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.768+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 30 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6fb9400 2024-04-27T04:35:04.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6fb9400 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f19c00 con 0x564dc6fb9400 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.769+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6fb8800 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 31 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6fb8800 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 32 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6fb8800 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 33 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.770+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 34 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707cc00 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 35 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d000 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 36 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d400 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 37 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.771+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c800 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 38 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016c00 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 39 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017800 con 0x564dc6fb8800 2024-04-27T04:35:04.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 40 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv14 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 41 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv14 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017c00 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 42 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv14 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.772+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6f18800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.773+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.773+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6f18800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.773+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.773+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6ac5c00 con 0x564dc6f18800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.774+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.774+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.774+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 43 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.774+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c000 con 0x564dc6fb8800 2024-04-27T04:35:04.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.774+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 44 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.774+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c400 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.775+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 45 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.775+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017000 con 0x564dc6f18800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.775+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.775+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016000 con 0x564dc6f18800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.775+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.775+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017400 con 0x564dc6f18800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016400 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 46 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8c00 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 47 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb8400 con 0x564dc6fb8800 2024-04-27T04:35:04.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64bea8b700 1 -- 172.21.15.191:0/4265309432 <== osd.3 v2:172.21.15.191:6800/41887 48 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6fb8800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.776+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9c00 con 0x564dc6f18800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.777+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv19 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.777+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9000 con 0x564dc6f18800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.777+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv19 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.777+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6fb9800 con 0x564dc6f18800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.777+0000 7f64bda89700 1 -- 172.21.15.191:0/4265309432 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv19 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6f18800 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.777+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18c00 con 0x564dc6fb9400 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.778+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.778+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc6f18400 con 0x564dc6fb9400 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.778+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 27 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.778+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707cc00 con 0x564dc6fb9400 2024-04-27T04:35:04.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.778+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 <== osd.2 v2:172.21.15.191:6816/41888 28 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6fb9400 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.779+0000 7f64a5258700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.779+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94892050235392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d000 con 0x564dc6ac5800 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.780+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94892050235392] v31'162 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.780+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94892048661248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d400 con 0x564dc6e55400 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.781+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94892048661248] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.781+0000 7f64c02e0a80 20 remove_watcher() i=0 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.781+0000 7f64c02e0a80 2 removed watcher, disabling cache 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.781+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94892048662400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c800 con 0x564dc6ac5800 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.782+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94892048662400] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.782+0000 7f64c02e0a80 20 remove_watcher() i=1 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.782+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94892048663552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707c000 con 0x564dc6e55c00 2024-04-27T04:35:04.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.783+0000 7f64be28a700 1 -- 172.21.15.191:0/423316132 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94892048663552] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05200 con 0x564dc6e55c00 2024-04-27T04:35:04.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.783+0000 7f64c02e0a80 20 remove_watcher() i=2 2024-04-27T04:35:04.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.783+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94892048664704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707d800 con 0x564dc6e55400 2024-04-27T04:35:04.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.784+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94892048664704] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f05440 con 0x564dc6e55400 2024-04-27T04:35:04.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.784+0000 7f64c02e0a80 20 remove_watcher() i=3 2024-04-27T04:35:04.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.784+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94892048665856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc707dc00 con 0x564dc6ac5800 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.785+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94892048665856] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6ac5800 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.785+0000 7f64c02e0a80 20 remove_watcher() i=4 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.785+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94892048667008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016c00 con 0x564dc6e55000 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.786+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94892048667008] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.786+0000 7f64c02e0a80 20 remove_watcher() i=5 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.786+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94892048668160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7017800 con 0x564dc6e55000 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.787+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94892048668160] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.787+0000 7f64c02e0a80 20 remove_watcher() i=6 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.787+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94892048669312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564dc7016800 con 0x564dc6e55000 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64bea8b700 1 -- 172.21.15.191:0/423316132 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94892048669312] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564dc6f04b40 con 0x564dc6e55000 2024-04-27T04:35:04.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 20 remove_watcher() i=7 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6e55000 msgr2=0x564dc6daf180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6e55000 0x564dc6daf180 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6ac5800 msgr2=0x564dc6dadb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6ac5800 0x564dc6dadb80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6e55400 msgr2=0x564dc6daec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6e55400 0x564dc6daec00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6e55c00 msgr2=0x564dc6dae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6e55c00 0x564dc6dae100 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54c00 msgr2=0x564dc6dad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54c00 0x564dc6dad600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x564dc7894030 tx=0x564dc6ef20c0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 msgr2=0x564dc6dacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/423316132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac5400 0x564dc6dacb00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x564dc793c6c0 tx=0x564dc6bb5ce0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 reap_dead start 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 shutdown_connections 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64bda89700 1 -- 172.21.15.191:0/423316132 reap_dead start 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 >> 172.21.15.191:0/423316132 conn(0x564dc6ac4c00 msgr2=0x564dc6bb1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 shutdown_connections 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.788+0000 7f64c02e0a80 1 -- 172.21.15.191:0/423316132 wait complete. 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6f18800 msgr2=0x564dc6fcc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x564dc6f18800 0x564dc6fcc580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6f18000 msgr2=0x564dc6fcc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x564dc6f18000 0x564dc6fcc000 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6fb9400 msgr2=0x564dc6fcd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x564dc6fb9400 0x564dc6fcd080 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6fb8800 msgr2=0x564dc6fccb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x564dc6fb8800 0x564dc6fccb00 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6fb8000 msgr2=0x564dc6daf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6fb8000 0x564dc6daf700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x564dc6e9d470 tx=0x564dc6ef2360 comp rx=0 tx=0).stop 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 msgr2=0x564dc6dae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x564dc6f39200 tx=0x564dc6ef24e0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64be28a700 1 -- 172.21.15.191:0/4265309432 reap_dead start 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 shutdown_connections 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/4265309432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6f19400 0x564dc6dae680 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 >> 172.21.15.191:0/4265309432 conn(0x564dc6f19000 msgr2=0x564dc6bb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 shutdown_connections 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/4265309432 wait complete. 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54000 msgr2=0x564dc6dad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54000 0x564dc6dad080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x564dc793d050 tx=0x564dc6e1ac60 comp rx=0 tx=0).stop 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 msgr2=0x564dc6dac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x564dc7895ec0 tx=0x564dc6bb5920 comp rx=0 tx=0).stop 2024-04-27T04:35:04.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 shutdown_connections 2024-04-27T04:35:04.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x564dc6e54000 0x564dc6dad080 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 --2- 172.21.15.191:0/2126549615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564dc6ac4800 0x564dc6dac580 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 >> 172.21.15.191:0/2126549615 conn(0x564dc6ac4400 msgr2=0x564dc6bb1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 shutdown_connections 2024-04-27T04:35:04.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.789+0000 7f64c02e0a80 1 -- 172.21.15.191:0/2126549615 wait complete. 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout:[ 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "tag": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.571437269045648694\u0000", 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:35:04.612949+0000", 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "objs": [ 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_1", 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.801 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_2", 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.802 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_3", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_4", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_5", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.803 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_6", 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "oid": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_7", 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "", 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: "instance": "" 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:04.804 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:04.805 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:04.805 INFO:teuthology.orchestra.run.smithi191.stdout:] 2024-04-27T04:35:04.805 DEBUG:tasks.util.rgw: json result: [{'tag': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.571437269045648694\x00', 'time': '2024-04-27T04:35:04.612949+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_7', 'key': '', 'instance': ''}]}] 2024-04-27T04:35:04.805 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-04-27T04:35:04.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-04-27T04:35:04.805 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-04-27T04:35:04.848 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:04.848 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:04.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.847+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.847+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.847+0000 7f0356464a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56192142ac00 0x561921712580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.847+0000 7f0356464a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561921755500 con 0x56192142a800 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.847+0000 7f0356464a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5619214cca00 con 0x56192142ac00 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f0353c0d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56192142ac00 0x561921712580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f035440e700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x56192142a800 0x561921516800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:33544/0 (socket says 172.21.15.191:33544) 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f035440e700 1 -- 172.21.15.191:0/2524548190 learned_addr learned my addr 172.21.15.191:0/2524548190 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f0353c0d700 1 -- 172.21.15.191:0/2524548190 >> v1:172.21.15.138:6789/0 conn(0x56192142a800 legacy=0x561921516800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f0353c0d700 1 -- 172.21.15.191:0/2524548190 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921453d40 con 0x56192142ac00 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f035340c700 1 -- 172.21.15.191:0/2524548190 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707676333 0 0) 0x5619221f8000 con 0x56192142a800 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2524548190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142ac00 0x561921712580 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x56192170d890 tx=0x56192151b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7442af75bc6d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f035340c700 1 -- 172.21.15.191:0/2524548190 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e34a0 con 0x56192142ac00 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f035340c700 1 -- 172.21.15.191:0/2524548190 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561921755800 con 0x56192142ac00 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.848+0000 7f035340c700 1 -- 172.21.15.191:0/2524548190 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e3600 con 0x56192142ac00 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 -- 172.21.15.191:0/2524548190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142ac00 msgr2=0x561921712580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 --2- 172.21.15.191:0/2524548190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142ac00 0x561921712580 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x56192170d890 tx=0x56192151b920 comp rx=0 tx=0).stop 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 -- 172.21.15.191:0/2524548190 shutdown_connections 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 --2- 172.21.15.191:0/2524548190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142ac00 0x561921712580 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 -- 172.21.15.191:0/2524548190 >> 172.21.15.191:0/2524548190 conn(0x56192142a400 msgr2=0x561921516000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 -- 172.21.15.191:0/2524548190 shutdown_connections 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 -- 172.21.15.191:0/2524548190 wait complete. 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.849+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f0356464a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f0356464a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619214cc500 con 0x56192142a800 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f035440e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f035440e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45948/0 (socket says 172.21.15.191:45948) 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f035440e700 1 -- 172.21.15.191:0/2031057266 learned_addr learned my addr 172.21.15.191:0/2031057266 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f035440e700 1 -- 172.21.15.191:0/2031057266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921453ba0 con 0x56192142a800 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f035440e700 1 --2- 172.21.15.191:0/2031057266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5619221fb770 tx=0x56192151bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6079b2f029eb07e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f0352c0b700 1 -- 172.21.15.191:0/2031057266 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e31e0 con 0x56192142a800 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f0352c0b700 1 -- 172.21.15.191:0/2031057266 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561921755b00 con 0x56192142a800 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.850+0000 7f0352c0b700 1 -- 172.21.15.191:0/2031057266 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e34a0 con 0x56192142a800 2024-04-27T04:35:04.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- 172.21.15.191:0/2031057266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 msgr2=0x561921712580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 --2- 172.21.15.191:0/2031057266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5619221fb770 tx=0x56192151bb60 comp rx=0 tx=0).stop 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- 172.21.15.191:0/2031057266 shutdown_connections 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 --2- 172.21.15.191:0/2031057266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- 172.21.15.191:0/2031057266 >> 172.21.15.191:0/2031057266 conn(0x56192142a400 msgr2=0x561921517000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- 172.21.15.191:0/2031057266 shutdown_connections 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- 172.21.15.191:0/2031057266 wait complete. 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f0356464a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619214cc3c0 con 0x56192142a800 2024-04-27T04:35:04.875 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f035440e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f035440e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45950/0 (socket says 172.21.15.191:45950) 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.851+0000 7f035440e700 1 -- 172.21.15.191:0/2875182908 learned_addr learned my addr 172.21.15.191:0/2875182908 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f035440e700 1 -- 172.21.15.191:0/2875182908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921453a00 con 0x56192142a800 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f035440e700 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5619221fbe30 tx=0x56192151ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f0351408700 1 -- 172.21.15.191:0/2875182908 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e3600 con 0x56192142a800 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f0351408700 1 -- 172.21.15.191:0/2875182908 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619214cbc80 con 0x56192142a800 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f0351408700 1 -- 172.21.15.191:0/2875182908 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e31e0 con 0x56192142a800 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561921453860 con 0x56192142a800 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.852+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619214536c0 con 0x56192142a800 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0356464a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0356464a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561921466640 con 0x56192142b400 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0351408700 1 -- 172.21.15.191:0/2875182908 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5619221ccb00 con 0x56192142a800 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0354c0f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0354c0f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45954/0 (socket says 172.21.15.191:45954) 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0354c0f700 1 -- 172.21.15.191:0/2514794577 learned_addr learned my addr 172.21.15.191:0/2514794577 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0351408700 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6000 0x561921713080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0351408700 1 -- 172.21.15.191:0/2875182908 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619214a5180 con 0x56192142a800 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f035440e700 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6000 0x561921713080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0354c0f700 1 -- 172.21.15.191:0/2514794577 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921452000 con 0x56192142b400 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2514794577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x5619222c1b60 tx=0x56192177ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4614ccd066d2f29b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f034ec03700 1 -- 172.21.15.191:0/2514794577 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e3600 con 0x56192142b400 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.853+0000 7f035440e700 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6000 0x561921713080 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5619222c0f30 tx=0x56192177e240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f034ec03700 1 -- 172.21.15.191:0/2514794577 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561921754300 con 0x56192142b400 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f034ec03700 1 -- 172.21.15.191:0/2514794577 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e31e0 con 0x56192142b400 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- 172.21.15.191:0/2514794577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 msgr2=0x561921712b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 --2- 172.21.15.191:0/2514794577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x5619222c1b60 tx=0x56192177ede0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- 172.21.15.191:0/2514794577 shutdown_connections 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 --2- 172.21.15.191:0/2514794577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- 172.21.15.191:0/2514794577 >> 172.21.15.191:0/2514794577 conn(0x56192142ac00 msgr2=0x561921517800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- 172.21.15.191:0/2514794577 shutdown_connections 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- 172.21.15.191:0/2514794577 wait complete. 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.854+0000 7f0356464a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619214668c0 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0354c0f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0354c0f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45970/0 (socket says 172.21.15.191:45970) 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 learned_addr learned my addr 172.21.15.191:0/2211763817 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921452ea0 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5619222c0450 tx=0x56192177e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f034d400700 1 -- 172.21.15.191:0/2211763817 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e3a20 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f034d400700 1 -- 172.21.15.191:0/2211763817 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619214cbc80 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561921452820 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f034d400700 1 -- 172.21.15.191:0/2211763817 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e3600 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.855+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619214529c0 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f034d400700 1 -- 172.21.15.191:0/2211763817 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5619221ccdc0 con 0x56192142b400 2024-04-27T04:35:04.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f034d400700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6c00 0x561921713600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f034d400700 1 -- 172.21.15.191:0/2211763817 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619214a5340 con 0x56192142b400 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6c00 0x561921713600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56192142b800 0x561921713b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x56192142b800 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.856+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56192142b800 0x561921713b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6c00 0x561921713600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x56192170d890 tx=0x56192151b800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56192142b800 0x561921713b80 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 realm 2024-04-27T04:35:04.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7000 con 0x56192142b800 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7400 con 0x56192142b800 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.857+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7800 con 0x56192142b800 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5619217c7c00 0x561921714100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7000 con 0x5619217c7c00 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f035440e700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5619217c7c00 0x561921714100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f035440e700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5619217c7c00 0x561921714100 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 20 searching for the correct realm 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.858+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7400 con 0x5619217c7c00 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f035440e700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5619217c7400 0x561921714c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7000 con 0x5619217c7400 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5619217c7400 0x561921714c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5619217c7400 0x561921714c00 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.882 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5619217c7000 0x561921715180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.859+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.860+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5619217c7000 0x561921715180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.860+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5619217c7000 0x561921715180 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.860+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.860+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.860+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.860+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7400 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.861+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.862+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7400 2024-04-27T04:35:04.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.863+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.864+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7400 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.865+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7400 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.866+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7c00 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7400 2024-04-27T04:35:04.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0356464a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0356464a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0356464a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0356464a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.867+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x56192142b800 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x56192142b800 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888800 con 0x5619217c7c00 2024-04-27T04:35:04.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 zone default found 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 4 Realm: () 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 zonegroup default 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x5619217c7000 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.868+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889000 con 0x5619217c7c00 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889400 con 0x5619217c7c00 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 started sync module instance, tier type = 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.869+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x5619217c7400 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.870+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.870+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94665935198976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888800 con 0x5619217c7400 2024-04-27T04:35:04.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.870+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889000 con 0x56192142b800 2024-04-27T04:35:04.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.871+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94665935198976] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.871+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.871+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94665935200128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889400 con 0x56192142b800 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.871+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x5619217c7c00 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.872+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.872+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94665935201280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x5619217c7c00 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.873+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x5619217c7400 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.873+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94665935200128] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.873+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.873+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94665935202432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619217c7800 con 0x5619217c7400 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.874+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x56192142b800 2024-04-27T04:35:04.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.874+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94665935201280] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861200 con 0x5619217c7c00 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.874+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.874+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94665935203584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x56192142b800 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.875+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888800 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.875+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94665935202432] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.876+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.876+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94665935204736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.876+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889000 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.876+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94665935203584] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.876+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94665935204736] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.877+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.877+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94665935205888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889400 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.877+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888000 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94665935207040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x5619217c7000 2024-04-27T04:35:04.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94665935205888] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=0 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=1 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=2 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=3 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=4 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=5 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.878+0000 7f0356464a80 20 add_watcher() i=6 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.879+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94665935207040] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.879+0000 7f0356464a80 20 add_watcher() i=7 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.879+0000 7f0356464a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.879+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888800 con 0x56192142b800 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.879+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.879+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94665936797696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x56192142b800 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94665936797696] v31'163 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f033b3dc700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0356464a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0356464a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0356464a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.880+0000 7f0356464a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619214cc3c0 con 0x561921889400 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f0353c0d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f0353c0d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45972/0 (socket says 172.21.15.191:45972) 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f0353c0d700 1 -- 172.21.15.191:0/637342105 learned_addr learned my addr 172.21.15.191:0/637342105 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f0353c0d700 1 -- 172.21.15.191:0/637342105 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921452b60 con 0x561921889400 2024-04-27T04:35:04.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f0353c0d700 1 --2- 172.21.15.191:0/637342105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5619217ced50 tx=0x56192151b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2df18305de7c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f033abdb700 1 -- 172.21.15.191:0/637342105 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619214e3760 con 0x561921889400 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f033abdb700 1 -- 172.21.15.191:0/637342105 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56192189e000 con 0x561921889400 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.881+0000 7f033abdb700 1 -- 172.21.15.191:0/637342105 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619218a0000 con 0x561921889400 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- 172.21.15.191:0/637342105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 msgr2=0x561921714680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 --2- 172.21.15.191:0/637342105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5619217ced50 tx=0x56192151b980 comp rx=0 tx=0).stop 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- 172.21.15.191:0/637342105 shutdown_connections 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 --2- 172.21.15.191:0/637342105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- 172.21.15.191:0/637342105 >> 172.21.15.191:0/637342105 conn(0x561921889000 msgr2=0x561921518000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- 172.21.15.191:0/637342105 shutdown_connections 2024-04-27T04:35:04.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- 172.21.15.191:0/637342105 wait complete. 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 Processor -- start 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- start start 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.882+0000 7f0356464a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619214cc500 con 0x561921889400 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0353c0d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0353c0d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45976/0 (socket says 172.21.15.191:45976) 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 learned_addr learned my addr 172.21.15.191:0/2283534830 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921452d00 con 0x561921889400 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x561921899260 tx=0x56192151b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f03393d8700 1 -- 172.21.15.191:0/2283534830 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619218a0160 con 0x561921889400 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f03393d8700 1 -- 172.21.15.191:0/2283534830 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56192189e180 con 0x561921889400 2024-04-27T04:35:04.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f03393d8700 1 -- 172.21.15.191:0/2283534830 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619218a02c0 con 0x561921889400 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561921452340 con 0x561921889400 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.883+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561921453d40 con 0x561921889400 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.884+0000 7f03393d8700 1 -- 172.21.15.191:0/2283534830 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5619221cdb80 con 0x561921889400 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.884+0000 7f03393d8700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561921916000 0x561921715700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.884+0000 7f03393d8700 1 -- 172.21.15.191:0/2283534830 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619214a5500 con 0x561921889400 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.884+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561921916000 0x561921715700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.884+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561921888000 0x561921934000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.884+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921888000 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.885+0000 7f035440e700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561921888000 0x561921934000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.885+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561921916000 0x561921715700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561921815ef0 tx=0x5619218ab260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.885+0000 7f035440e700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561921888000 0x561921934000 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.886+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'52 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.886+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561921888800 0x561921934580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.886+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921888800 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.886+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561921888800 0x561921934580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.886+0000 7f0353c0d700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561921888800 0x561921934580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.887+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'25 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.887+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561921916800 0x561921934b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.887+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916c00 con 0x561921916800 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.887+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561921916800 0x561921934b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.888+0000 7f0354c0f700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561921916800 0x561921934b00 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.888+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.888+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921888000 2024-04-27T04:35:04.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.889+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'164 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.889+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561921917400 0x561921935080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.889+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921917400 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.889+0000 7f035440e700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561921917400 0x561921935080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.890+0000 7f035440e700 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561921917400 0x561921935080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.891+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'56 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.891+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921916800 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.892+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'44 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.892+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921888000 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.893+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.893+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921888800 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.893+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'39 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.893+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921917400 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.894+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.894+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921916800 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.895+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.895+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921917400 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.896+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'57 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.896+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921888800 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.897+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'22 uv20 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.897+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921917400 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.898+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.898+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195ec00 con 0x561921888800 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.899+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'53 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.899+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921888000 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.900+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'165 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.900+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f400 con 0x561921916800 2024-04-27T04:35:04.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.901+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'47 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.901+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921917400 2024-04-27T04:35:04.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.902+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'58 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.902+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921916800 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.903+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'31 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.903+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921888000 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.904+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.904+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921888000 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.905+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'166 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.905+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921916800 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.906+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'48 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.906+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921917400 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.907+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'35 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.907+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.908+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'27 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.908+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.908+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'37 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.908+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.909+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.909+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.910+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'45 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.910+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921888800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.911+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.912+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f400 con 0x561921916800 2024-04-27T04:35:04.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.912+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'31 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.912+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921888800 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.913+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'40 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.913+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921916800 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.914+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'49 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.914+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921888800 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.915+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'54 uv19 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.915+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921917400 2024-04-27T04:35:04.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.916+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f0336bd3700 20 reqs_thread_entry: start 2024-04-27T04:35:04.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f0356464a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:04.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f0356464a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:04.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f0334bcf700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:35 2024 2024-04-27T04:35:04.919 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:04.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f03343ad700 20 reqs_thread_entry: start 2024-04-27T04:35:04.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-04-27T04:35:04.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.917+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:33 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921888000 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.918+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(33 gc.3 [call] v31'167 uv167 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.918+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:34 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921888000 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(34 gc.3 [call out=15b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:35 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921888000 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(35 gc.3 [call out=24b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921888000 2024-04-27T04:35:04.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(36 gc.3 [call out=15b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:37 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921888000 2024-04-27T04:35:04.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(37 gc.3 [call out=15b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.919+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921888000 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.920+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(38 gc.3 [call] v31'168 uv168 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.920+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.920+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:39 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921917400 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.921+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(39 gc.4 [call] v31'59 uv59 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.921+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:40 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921917400 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(40 gc.4 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:41 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921917400 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(41 gc.4 [call out=24b] v0'0 uv59 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:42 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921917400 2024-04-27T04:35:04.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(42 gc.4 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:43 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921917400 2024-04-27T04:35:04.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(43 gc.4 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.922+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921917400 2024-04-27T04:35:04.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.923+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(44 gc.4 [call] v31'60 uv60 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.923+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-04-27T04:35:04.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.923+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:45 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921916800 2024-04-27T04:35:04.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(45 gc.5 [call] v31'46 uv46 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:46 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(46 gc.5 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:47 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(47 gc.5 [call out=24b] v0'0 uv46 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.924+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.925+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.925+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.925+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(49 gc.5 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.925+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.925+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921916800 2024-04-27T04:35:04.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.926+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(50 gc.5 [call] v31'47 uv47 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.926+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-04-27T04:35:04.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.926+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921888000 2024-04-27T04:35:04.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.927+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(51 gc.6 [call] v31'28 uv28 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.927+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.927+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(52 gc.6 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.927+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.927+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f400 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(53 gc.6 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:54 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(54 gc.6 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:55 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(55 gc.6 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.928+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:56 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921888000 2024-04-27T04:35:04.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.929+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(56 gc.6 [call] v31'29 uv29 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921888000 2024-04-27T04:35:04.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.929+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-04-27T04:35:04.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.929+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921888800 2024-04-27T04:35:04.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.930+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(57 gc.7 [call] v31'41 uv41 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.930+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.930+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(58 gc.7 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.930+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.930+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f400 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(59 gc.7 [call out=24b] v0'0 uv41 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:60 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(60 gc.7 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:61 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(61 gc.7 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.931+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921888800 2024-04-27T04:35:04.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.932+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(62 gc.7 [call] v31'42 uv42 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.932+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-04-27T04:35:04.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.932+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195ec00 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.933+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(63 gc.8 [call] v31'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.933+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.933+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(64 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.933+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.933+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(65 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(66 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.939 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921917400 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(67 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.934+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921917400 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.935+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(68 gc.8 [call] v31'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.935+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.935+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921916800 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(69 gc.9 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921916800 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(70 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921916800 2024-04-27T04:35:04.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(71 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.936+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:72 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.937+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(72 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.937+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.937+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.937+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(73 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.937+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.937+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:74 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db800 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.938+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(74 gc.9 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.938+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-04-27T04:35:04.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.938+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:75 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921917400 2024-04-27T04:35:04.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(75 gc.10 [call] v31'61 uv61 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921917400 2024-04-27T04:35:04.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== osd_op_reply(76 gc.10 [call out=15b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(77 gc.10 [call out=24b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(78 gc.10 [call out=15b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:79 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(79 gc.10 [call out=15b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.939+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:80 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921917400 2024-04-27T04:35:04.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.940+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(80 gc.10 [call] v31'62 uv62 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861200 con 0x561921917400 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.940+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.940+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.941+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(81 gc.11 [call] v31'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.941+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:82 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(82 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:83 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(83 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(84 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(85 gc.11 [call out=1700b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+1700 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 20 garbage collection: RGWGC::process iterating over entry tag='e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.571437269045648694', time=2024-04-27T04:35:04.612949+0000, chain.objs.size()=7 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_1 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:86 3.30s0 3:0d1cf2cf:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_1:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921916800 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.942+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_2 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:87 3.3s0 3:c3215e64:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_2:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195ec00 con 0x561921917400 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_3 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:88 3.cs0 3:30a148fa:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_3:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921916800 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_4 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:89 3.2ds0 3:b6a32118:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_4:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921916800 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_5 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:90 3.20s0 3:04aa5ace:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_5:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921917400 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_6 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 3.10s0 3:0939e6da:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_6:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921888000 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_7 2024-04-27T04:35:04.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.943+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:92 3.3s0 3:c08336d4:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_7:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921917400 2024-04-27T04:35:04.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.947+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(86 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_1 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.947+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(89 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_4 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.950+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(90 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_5 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.950+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 27 ==== osd_op_reply(87 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_2 [call] v31'3 uv3 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.950+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(88 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_3 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.951+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 28 ==== osd_op_reply(92 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_7 [call] v31'4 uv4 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.951+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(91 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.1__shadow_.oiLvYNLQqTdLWFmRMC2l4EysRhR2Gd8_6 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.951+0000 7f0356464a80 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-27T04:35:04.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.951+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:93 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921888800 2024-04-27T04:35:04.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.955+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(93 gc.11 [call] v31'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.956 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.955+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:94 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921888800 2024-04-27T04:35:04.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.958+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(94 gc.11 [call] v31'25 uv25 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.958+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-04-27T04:35:04.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.958+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 29 ==== osd_op_reply(95 gc.12 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:96 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 30 ==== osd_op_reply(96 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:97 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 31 ==== osd_op_reply(97 gc.12 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 32 ==== osd_op_reply(98 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.959+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:99 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921917400 2024-04-27T04:35:04.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.960+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 33 ==== osd_op_reply(99 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.960+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.960+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:100 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921917400 2024-04-27T04:35:04.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.960+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 34 ==== osd_op_reply(100 gc.12 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.960+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-04-27T04:35:04.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.960+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921888800 2024-04-27T04:35:04.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(101 gc.13 [call] v31'55 uv55 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.963 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921888800 2024-04-27T04:35:04.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(102 gc.13 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.964 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:103 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921888800 2024-04-27T04:35:04.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(103 gc.13 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.962+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:104 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195ec00 con 0x561921888800 2024-04-27T04:35:04.965 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.963+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(104 gc.13 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.963+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.963+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:105 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921888800 2024-04-27T04:35:04.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.963+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(105 gc.13 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.963+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.966 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.963+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:106 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f400 con 0x561921888800 2024-04-27T04:35:04.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.964+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(106 gc.13 [call] v31'56 uv56 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:04.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.964+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.964+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:107 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.965+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(107 gc.14 [call] v31'169 uv169 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.965+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:108 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(108 gc.14 [call out=15b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:109 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(109 gc.14 [call out=24b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:110 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(110 gc.14 [call out=15b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:111 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(111 gc.14 [call out=15b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.966+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:112 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921888000 2024-04-27T04:35:04.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.967+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(112 gc.14 [call] v31'170 uv170 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.967+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.967+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:113 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(113 gc.15 [call] v31'50 uv50 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:114 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 29 ==== osd_op_reply(114 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:115 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 30 ==== osd_op_reply(115 gc.15 [call out=24b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.968+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:116 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.969+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 31 ==== osd_op_reply(116 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.969+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.969+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:117 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.969+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 32 ==== osd_op_reply(117 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.969+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.969+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:118 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921916800 2024-04-27T04:35:04.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.970+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 33 ==== osd_op_reply(118 gc.15 [call] v31'51 uv51 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.970+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-04-27T04:35:04.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.970+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:119 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921917400 2024-04-27T04:35:04.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.971+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 35 ==== osd_op_reply(119 gc.16 [call] v31'63 uv63 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.971+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:120 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921917400 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.971+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 36 ==== osd_op_reply(120 gc.16 [call out=15b] v0'0 uv63 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.971+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.971+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:121 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921917400 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 37 ==== osd_op_reply(121 gc.16 [call out=24b] v0'0 uv63 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:122 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921917400 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 38 ==== osd_op_reply(122 gc.16 [call out=15b] v0'0 uv63 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:04.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:123 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921917400 2024-04-27T04:35:04.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 39 ==== osd_op_reply(123 gc.16 [call out=15b] v0'0 uv63 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:04.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.972+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:124 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921917400 2024-04-27T04:35:04.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.973+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 40 ==== osd_op_reply(124 gc.16 [call] v31'64 uv64 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:04.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.973+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-04-27T04:35:04.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.973+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:125 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921916800 2024-04-27T04:35:04.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 34 ==== osd_op_reply(125 gc.17 [call] v31'32 uv32 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:04.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:126 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921916800 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 35 ==== osd_op_reply(126 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:127 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921916800 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 36 ==== osd_op_reply(127 gc.17 [call out=24b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.974+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:128 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921916800 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.975+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 37 ==== osd_op_reply(128 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.975+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.975+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:129 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921916800 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.975+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 38 ==== osd_op_reply(129 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.975+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.975+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:130 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921916800 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.976+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 39 ==== osd_op_reply(130 gc.17 [call] v31'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.976+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.976+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:131 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921888000 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.977+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(131 gc.18 [call] v31'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.977+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:132 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921888000 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.977+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(132 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.977+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.977+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:133 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921888000 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(133 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:134 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(134 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:135 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(135 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.978+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:136 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.979+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(136 gc.18 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.979+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.979+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:137 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db800 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.980+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(137 gc.19 [call] v31'171 uv171 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.980+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:138 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(138 gc.19 [call out=15b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:139 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 34 ==== osd_op_reply(139 gc.19 [call out=24b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:140 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 35 ==== osd_op_reply(140 gc.19 [call out=15b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:141 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 36 ==== osd_op_reply(141 gc.19 [call out=15b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.981+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:142 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.982+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 37 ==== osd_op_reply(142 gc.19 [call] v31'172 uv172 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.982+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.982+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:143 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921916800 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 40 ==== osd_op_reply(143 gc.20 [call] v31'52 uv52 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:144 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 41 ==== osd_op_reply(144 gc.20 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:145 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 42 ==== osd_op_reply(145 gc.20 [call out=24b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.983+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:146 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.984+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 43 ==== osd_op_reply(146 gc.20 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.984+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.984+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:147 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.984+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 44 ==== osd_op_reply(147 gc.20 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.984+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.984+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:148 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921916800 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.985+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 45 ==== osd_op_reply(148 gc.20 [call] v31'53 uv53 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.985+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.985+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:149 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 41 ==== osd_op_reply(149 gc.21 [call] v31'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:150 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 42 ==== osd_op_reply(150 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:151 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 43 ==== osd_op_reply(151 gc.21 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.986+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:152 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.987+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 44 ==== osd_op_reply(152 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.987+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.987+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:153 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921917400 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.987+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 45 ==== osd_op_reply(153 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.987+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.987+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:154 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921917400 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.988+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 46 ==== osd_op_reply(154 gc.21 [call] v31'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.988+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.988+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:155 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921916800 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 46 ==== osd_op_reply(155 gc.22 [call] v31'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:156 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921916800 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 47 ==== osd_op_reply(156 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:157 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921916800 2024-04-27T04:35:05.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 48 ==== osd_op_reply(157 gc.22 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.989+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:158 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.990+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 49 ==== osd_op_reply(158 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.990+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.990+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:159 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.990+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 50 ==== osd_op_reply(159 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.990+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.990+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.991+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 51 ==== osd_op_reply(160 gc.22 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.991+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.991+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:161 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db800 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 52 ==== osd_op_reply(161 gc.23 [call] v31'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:162 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921916800 2024-04-27T04:35:05.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 53 ==== osd_op_reply(162 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 54 ==== osd_op_reply(163 gc.23 [call out=24b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 55 ==== osd_op_reply(164 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.992+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.993+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 56 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.993+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.993+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:166 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.993+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 57 ==== osd_op_reply(166 gc.23 [call] v31'39 uv39 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.993+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.994+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921916800 2024-04-27T04:35:05.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.994+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 58 ==== osd_op_reply(167 gc.24 [call] v31'22 uv22 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.994+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 59 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:169 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 60 ==== osd_op_reply(169 gc.24 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:170 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 61 ==== osd_op_reply(170 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:171 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 62 ==== osd_op_reply(171 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.995+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:172 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.996+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 63 ==== osd_op_reply(172 gc.24 [call] v31'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.996+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.996+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:173 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 64 ==== osd_op_reply(173 gc.25 [call] v31'48 uv48 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:174 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 65 ==== osd_op_reply(174 gc.25 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:175 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 66 ==== osd_op_reply(175 gc.25 [call out=24b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.997+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:176 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.998+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 67 ==== osd_op_reply(176 gc.25 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.998+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.998+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:177 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921916800 2024-04-27T04:35:05.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.998+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 68 ==== osd_op_reply(177 gc.25 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.998+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.998+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:178 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921916800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.999+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 69 ==== osd_op_reply(178 gc.25 [call] v31'49 uv49 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.999+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:04.999+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:179 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921888800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(179 gc.26 [call] v31'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:180 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921888800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(180 gc.26 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:181 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921888800 2024-04-27T04:35:05.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== osd_op_reply(181 gc.26 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.000+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:182 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.001+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== osd_op_reply(182 gc.26 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.001+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.001+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:183 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.001+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== osd_op_reply(183 gc.26 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.001+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.001+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:184 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db800 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.002+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(184 gc.26 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.002+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.002+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:185 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dbc00 con 0x561921916800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.003+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 70 ==== osd_op_reply(185 gc.27 [call] v31'32 uv32 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.003+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:186 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da400 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.003+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 71 ==== osd_op_reply(186 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.003+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.003+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:187 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da800 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 72 ==== osd_op_reply(187 gc.27 [call out=24b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:188 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219dac00 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 73 ==== osd_op_reply(188 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:189 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219da000 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 74 ==== osd_op_reply(189 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.004+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:190 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x561921916800 2024-04-27T04:35:05.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.005+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 75 ==== osd_op_reply(190 gc.27 [call] v31'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.005+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.005+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:191 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921888800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 33 ==== osd_op_reply(191 gc.28 [call] v31'43 uv43 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:192 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921888800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 34 ==== osd_op_reply(192 gc.28 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:193 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921888800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 35 ==== osd_op_reply(193 gc.28 [call out=24b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.006+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:194 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921888800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.007+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 36 ==== osd_op_reply(194 gc.28 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.007+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.007+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:195 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921888800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.007+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 37 ==== osd_op_reply(195 gc.28 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.007+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.007+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:196 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921888800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.008+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 38 ==== osd_op_reply(196 gc.28 [call] v31'44 uv44 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.008+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.008+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:197 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889c00 con 0x561921916800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 76 ==== osd_op_reply(197 gc.29 [call] v31'54 uv54 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:198 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888c00 con 0x561921916800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 77 ==== osd_op_reply(198 gc.29 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:199 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921888400 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 78 ==== osd_op_reply(199 gc.29 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.009+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:200 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921889800 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.010+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 79 ==== osd_op_reply(200 gc.29 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.010+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.010+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:201 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192142bc00 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.010+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 80 ==== osd_op_reply(201 gc.29 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.010+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.010+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:202 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.011+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 81 ==== osd_op_reply(202 gc.29 [call] v31'55 uv55 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.011+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.011+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:203 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x561921888800 2024-04-27T04:35:05.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.012+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 39 ==== osd_op_reply(203 gc.30 [call] v31'57 uv57 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.012+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:204 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917000 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.012+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 40 ==== osd_op_reply(204 gc.30 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.012+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.012+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:205 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db000 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 41 ==== osd_op_reply(205 gc.30 [call out=24b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:206 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db400 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 42 ==== osd_op_reply(206 gc.30 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:207 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219db800 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 43 ==== osd_op_reply(207 gc.30 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.013+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:208 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.014+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 44 ==== osd_op_reply(208 gc.30 [call] v31'58 uv58 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.014+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-04-27T04:35:05.033 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.014+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:209 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e400 con 0x561921917400 2024-04-27T04:35:05.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.015+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 47 ==== osd_op_reply(209 gc.31 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.015+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:210 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e800 con 0x561921917400 2024-04-27T04:35:05.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 48 ==== osd_op_reply(210 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:211 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195ec00 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 49 ==== osd_op_reply(211 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:212 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f000 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 50 ==== osd_op_reply(212 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:213 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f400 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 51 ==== osd_op_reply(213 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.016+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:214 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195f800 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.017+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.2 v2:172.21.15.191:6816/41888 52 ==== osd_op_reply(214 gc.31 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x561921861680 con 0x561921917400 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.017+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.017+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:215 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195fc00 con 0x561921888000 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.018+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 38 ==== osd_op_reply(215 gc.0 [call] v31'53 uv53 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.018+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:216 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e2000 con 0x561921888000 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.018+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 39 ==== osd_op_reply(216 gc.0 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:217 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e2400 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 40 ==== osd_op_reply(217 gc.0 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:218 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e2800 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 41 ==== osd_op_reply(218 gc.0 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:219 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e2c00 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 42 ==== osd_op_reply(219 gc.0 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.019+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:220 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3000 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.020+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 <== osd.1 v2:172.21.15.191:6808/41886 43 ==== osd_op_reply(220 gc.0 [call] v31'54 uv54 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861680 con 0x561921888000 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.020+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.020+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:221 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3400 con 0x561921888800 2024-04-27T04:35:05.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.021+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 45 ==== osd_op_reply(221 gc.1 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.021+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:222 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3800 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 46 ==== osd_op_reply(222 gc.1 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:223 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3c00 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 47 ==== osd_op_reply(223 gc.1 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:224 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e4000 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 48 ==== osd_op_reply(224 gc.1 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:225 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e4400 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 49 ==== osd_op_reply(225 gc.1 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.022+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:226 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e4800 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.024+0000 7f0353c0d700 1 -- 172.21.15.191:0/2283534830 <== osd.0 v2:172.21.15.138:6800/39496 50 ==== osd_op_reply(226 gc.1 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921861440 con 0x561921888800 2024-04-27T04:35:05.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.024+0000 7f0356464a80 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.024+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:227 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e4c00 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 82 ==== osd_op_reply(227 gc.2 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:228 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e5000 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 83 ==== osd_op_reply(228 gc.2 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:229 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e5400 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 84 ==== osd_op_reply(229 gc.2 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:230 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e5800 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 85 ==== osd_op_reply(230 gc.2 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:231 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e5c00 con 0x561921916800 2024-04-27T04:35:05.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 86 ==== osd_op_reply(231 gc.2 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.025+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:232 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e4c00 con 0x561921916800 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.026+0000 7f0354c0f700 1 -- 172.21.15.191:0/2283534830 <== osd.3 v2:172.21.15.191:6800/41887 87 ==== osd_op_reply(232 gc.2 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x561921860b40 con 0x561921916800 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.027+0000 7f033b3dc700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.027+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94665936797696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e5000 con 0x56192142b800 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.028+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94665936797696] v31'173 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.028+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94665935198976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e4000 con 0x5619217c7400 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.029+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94665935198976] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:05.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.029+0000 7f0356464a80 20 remove_watcher() i=0 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.029+0000 7f0356464a80 2 removed watcher, disabling cache 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.029+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94665935200128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3400 con 0x56192142b800 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.030+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94665935200128] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.030+0000 7f0356464a80 20 remove_watcher() i=1 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.030+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94665935201280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3800 con 0x5619217c7c00 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.031+0000 7f035440e700 1 -- 172.21.15.191:0/2211763817 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94665935201280] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861680 con 0x5619217c7c00 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.031+0000 7f0356464a80 20 remove_watcher() i=2 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.031+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94665935202432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619219e3c00 con 0x5619217c7400 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.032+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94665935202432] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921861440 con 0x5619217c7400 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.032+0000 7f0356464a80 20 remove_watcher() i=3 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.032+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94665935203584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56192195e000 con 0x56192142b800 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.033+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94665935203584] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x56192142b800 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.033+0000 7f0356464a80 20 remove_watcher() i=4 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.033+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94665935204736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917c00 con 0x5619217c7000 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.034+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94665935204736] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:05.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.034+0000 7f0356464a80 20 remove_watcher() i=5 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.034+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94665935205888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921916400 con 0x5619217c7000 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.035+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94665935205888] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.035+0000 7f0356464a80 20 remove_watcher() i=6 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.035+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94665935207040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561921917800 con 0x5619217c7000 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0354c0f700 1 -- 172.21.15.191:0/2211763817 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94665935207040] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561921860b40 con 0x5619217c7000 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 20 remove_watcher() i=7 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5619217c7000 msgr2=0x561921715180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5619217c7000 0x561921715180 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56192142b800 msgr2=0x561921713b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56192142b800 0x561921713b80 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5619217c7400 msgr2=0x561921714c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5619217c7400 0x561921714c00 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5619217c7c00 msgr2=0x561921714100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5619217c7c00 0x561921714100 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6c00 msgr2=0x561921713600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6c00 0x561921713600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x56192170d890 tx=0x56192151b800 comp rx=0 tx=0).stop 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 msgr2=0x561921712b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 --2- 172.21.15.191:0/2211763817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142b400 0x561921712b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5619222c0450 tx=0x56192177e840 comp rx=0 tx=0).stop 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 reap_dead start 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0353c0d700 1 -- 172.21.15.191:0/2211763817 reap_dead start 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 shutdown_connections 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 >> 172.21.15.191:0/2211763817 conn(0x56192142ac00 msgr2=0x561921517800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 shutdown_connections 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.036+0000 7f0356464a80 1 -- 172.21.15.191:0/2211763817 wait complete. 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561921888800 msgr2=0x561921934580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561921888800 0x561921934580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561921888000 msgr2=0x561921934000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561921888000 0x561921934000 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561921917400 msgr2=0x561921935080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561921917400 0x561921935080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561921916800 msgr2=0x561921934b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561921916800 0x561921934b00 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561921916000 msgr2=0x561921715700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561921916000 0x561921715700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561921815ef0 tx=0x5619218ab260 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 msgr2=0x561921714680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f035440e700 1 -- 172.21.15.191:0/2283534830 reap_dead start 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x561921899260 tx=0x56192151b8c0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 shutdown_connections 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2283534830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561921889400 0x561921714680 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 >> 172.21.15.191:0/2283534830 conn(0x561921889000 msgr2=0x561921518000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 shutdown_connections 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2283534830 wait complete. 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6000 msgr2=0x561921713080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6000 0x561921713080 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5619222c0f30 tx=0x56192177e240 comp rx=0 tx=0).stop 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 msgr2=0x561921712580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5619221fbe30 tx=0x56192151ba40 comp rx=0 tx=0).stop 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 shutdown_connections 2024-04-27T04:35:05.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5619217c6000 0x561921713080 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 --2- 172.21.15.191:0/2875182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56192142a800 0x561921712580 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 >> 172.21.15.191:0/2875182908 conn(0x56192142a400 msgr2=0x561921517000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 shutdown_connections 2024-04-27T04:35:05.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.037+0000 7f0356464a80 1 -- 172.21.15.191:0/2875182908 wait complete. 2024-04-27T04:35:05.057 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-27T04:35:05.057 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-27T04:35:05.058 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-27T04:35:05.133 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:05.133 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:05.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.134 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4d40fa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565117142c00 0x56511742a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4d40fa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56511746d500 con 0x565117142800 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4d40fa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5651171e4a00 con 0x565117142c00 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4abb8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565117142c00 0x56511742a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4b3b9700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x565117142800 0x56511722e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:33550/0 (socket says 172.21.15.191:33550) 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4abb8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565117142c00 0x56511742a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45978/0 (socket says 172.21.15.191:45978) 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.132+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1976874345 learned_addr learned my addr 172.21.15.191:0/1976874345 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.133+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1976874345 >> v1:172.21.15.138:6789/0 conn(0x565117142800 legacy=0x56511722e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.133+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1976874345 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716bd40 con 0x565117142c00 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.133+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/1976874345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142c00 0x56511742a580 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x565117425890 tx=0x565117233920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26315da44d5e4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.133+0000 7f3d4a3b7700 1 -- 172.21.15.191:0/1976874345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb4a0 con 0x565117142c00 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.133+0000 7f3d4a3b7700 1 -- 172.21.15.191:0/1976874345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56511746d800 con 0x565117142c00 2024-04-27T04:35:05.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.133+0000 7f3d4a3b7700 1 -- 172.21.15.191:0/1976874345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb600 con 0x565117142c00 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1976874345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142c00 msgr2=0x56511742a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1976874345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142c00 0x56511742a580 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x565117425890 tx=0x565117233920 comp rx=0 tx=0).stop 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1976874345 shutdown_connections 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1976874345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142c00 0x56511742a580 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1976874345 >> 172.21.15.191:0/1976874345 conn(0x565117142400 msgr2=0x56511722e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1976874345 shutdown_connections 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1976874345 wait complete. 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.134+0000 7f3d4d40fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651171e4500 con 0x565117142800 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4b3b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4b3b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45980/0 (socket says 172.21.15.191:45980) 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/644326925 learned_addr learned my addr 172.21.15.191:0/644326925 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/644326925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716bba0 con 0x565117142800 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/644326925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x565117f1b800 tx=0x565117233b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcd5ba6334311da server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d49bb6700 1 -- 172.21.15.191:0/644326925 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb4a0 con 0x565117142800 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d49bb6700 1 -- 172.21.15.191:0/644326925 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56511746db00 con 0x565117142800 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d49bb6700 1 -- 172.21.15.191:0/644326925 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb600 con 0x565117142800 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/644326925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 msgr2=0x56511742a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/644326925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x565117f1b800 tx=0x565117233b60 comp rx=0 tx=0).stop 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/644326925 shutdown_connections 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/644326925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/644326925 >> 172.21.15.191:0/644326925 conn(0x565117142400 msgr2=0x56511722f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/644326925 shutdown_connections 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.135+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/644326925 wait complete. 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4d40fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4d40fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651171e43c0 con 0x565117142800 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4b3b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4b3b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:45984/0 (socket says 172.21.15.191:45984) 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/922301661 learned_addr learned my addr 172.21.15.191:0/922301661 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/922301661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716ba00 con 0x565117142800 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.136+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x565117f1ba40 tx=0x565117233a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d483b3700 1 -- 172.21.15.191:0/922301661 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb8c0 con 0x565117142800 2024-04-27T04:35:05.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d483b3700 1 -- 172.21.15.191:0/922301661 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651171e3c80 con 0x565117142800 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56511716b860 con 0x565117142800 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d483b3700 1 -- 172.21.15.191:0/922301661 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb4a0 con 0x565117142800 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56511716b6c0 con 0x565117142800 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4d40fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4d40fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651171e4000 con 0x565117143400 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d483b3700 1 -- 172.21.15.191:0/922301661 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565117ee4b00 con 0x565117142800 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.137+0000 7f3d4bbba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4bbba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46000/0 (socket says 172.21.15.191:46000) 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4bbba700 1 -- 172.21.15.191:0/4172330310 learned_addr learned my addr 172.21.15.191:0/4172330310 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d483b3700 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8000 0x56511742b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d483b3700 1 -- 172.21.15.191:0/922301661 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5651171bd180 con 0x565117142800 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8000 0x56511742b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4bbba700 1 -- 172.21.15.191:0/4172330310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716a000 con 0x565117143400 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/4172330310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5651174d6060 tx=0x565117490de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed37df9e29c2932 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8000 0x56511742b080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x565117fc2fc0 tx=0x565117490240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d45bae700 1 -- 172.21.15.191:0/4172330310 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb8c0 con 0x565117143400 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d45bae700 1 -- 172.21.15.191:0/4172330310 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56511746c300 con 0x565117143400 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d45bae700 1 -- 172.21.15.191:0/4172330310 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb4a0 con 0x565117143400 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/4172330310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 msgr2=0x56511742ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/4172330310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5651174d6060 tx=0x565117490de0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/4172330310 shutdown_connections 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/4172330310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/4172330310 >> 172.21.15.191:0/4172330310 conn(0x565117142c00 msgr2=0x56511722f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.138+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/4172330310 shutdown_connections 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/4172330310 wait complete. 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4d40fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4d40fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511717e640 con 0x565117143400 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4bbba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4bbba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46002/0 (socket says 172.21.15.191:46002) 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 learned_addr learned my addr 172.21.15.191:0/2462756345 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716aea0 con 0x565117143400 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.139+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x565117fc2930 tx=0x565117490840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d443ab700 1 -- 172.21.15.191:0/2462756345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fbb80 con 0x565117143400 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d443ab700 1 -- 172.21.15.191:0/2462756345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651171e3c80 con 0x565117143400 2024-04-27T04:35:05.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56511716a820 con 0x565117143400 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d443ab700 1 -- 172.21.15.191:0/2462756345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb8c0 con 0x565117143400 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56511716a9c0 con 0x565117143400 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d443ab700 1 -- 172.21.15.191:0/2462756345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565117ee4dc0 con 0x565117143400 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d443ab700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8c00 0x56511742b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.140+0000 7f3d443ab700 1 -- 172.21.15.191:0/2462756345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6061+0+0 (secure 0 0 0) 0x5651171bd340 con 0x565117143400 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8c00 0x56511742b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565117143800 0x56511742bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565117143c00 con 0x565117143800 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565117143800 0x56511742bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8c00 0x56511742b600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x565117425890 tx=0x565117233800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565117143800 0x56511742bb80 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 20 realm 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9000 con 0x565117143800 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.141+0000 7f3d4d40fa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9400 con 0x565117143800 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9800 con 0x565117143800 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5651174d9c00 0x56511742c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9000 con 0x5651174d9c00 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5651174d9c00 0x56511742c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.142+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5651174d9c00 0x56511742c100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4d40fa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4d40fa80 20 searching for the correct realm 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9400 con 0x5651174d9c00 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5651174d9400 0x56511742cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9000 con 0x5651174d9400 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5651174d9400 0x56511742cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.143+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5651174d9400 0x56511742cc00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5651174d9000 0x56511742d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5651174d9000 0x56511742d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5651174d9000 0x56511742d180 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.144+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.145+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9400 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.146+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.146+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.146+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.146+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.146+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.146+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9400 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.147+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.148+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9400 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.149+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9400 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9c00 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.150+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9400 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565117143c00 con 0x565117143800 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.151+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a400 con 0x565117143800 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a800 con 0x5651174d9c00 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 zone default found 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 4 Realm: () 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 zonegroup default 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759ac00 con 0x5651174d9000 2024-04-27T04:35:05.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b000 con 0x5651174d9c00 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.152+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b400 con 0x5651174d9c00 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.153+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.153+0000 7f3d4d40fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.153+0000 7f3d4d40fa80 20 started sync module instance, tier type = 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.153+0000 7f3d4d40fa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.153+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b800 con 0x5651174d9400 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.154+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(44 notify.0 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.154+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94906282547968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a800 con 0x5651174d9400 2024-04-27T04:35:05.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.154+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b000 con 0x565117143800 2024-04-27T04:35:05.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.155+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94906282547968] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.155+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.155+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94906282549120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b400 con 0x565117143800 2024-04-27T04:35:05.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.155+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a400 con 0x5651174d9c00 2024-04-27T04:35:05.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.156+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94906282549120] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.159+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.159+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94906282550272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759ac00 con 0x5651174d9c00 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.159+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x565117143c00 con 0x5651174d9400 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.160+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(50 notify.3 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.160+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94906282551424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5651174d9800 con 0x5651174d9400 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.160+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x565117143800 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.160+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94906282550272] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.161+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.161+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94906282552576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b800 con 0x565117143800 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.161+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94906282551424] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.161+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a800 con 0x5651174d9000 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.162+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 notify.5 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.162+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94906282552576] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.162+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94906282553728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759bc00 con 0x5651174d9000 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.162+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b000 con 0x5651174d9000 2024-04-27T04:35:05.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.163+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94906282553728] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.163+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 notify.6 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.163+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94906282554880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b400 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.163+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a000 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94906282556032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759b800 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=0 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=1 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=2 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=3 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=4 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=5 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94906282554880] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.166+0000 7f3d4d40fa80 20 add_watcher() i=6 2024-04-27T04:35:05.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.167+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94906282556032] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.167+0000 7f3d4d40fa80 20 add_watcher() i=7 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.167+0000 7f3d4d40fa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.167+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759a800 con 0x565117143800 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.167+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.167+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94906284146688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511759bc00 con 0x565117143800 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.169+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94906284146688] v31'174 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.169+0000 7f3d32387700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.169+0000 7f3d4d40fa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.169+0000 7f3d4d40fa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4d40fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4d40fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511717e780 con 0x56511759b400 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4abb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4abb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46010/0 (socket says 172.21.15.191:46010) 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.170+0000 7f3d4abb8700 1 -- 172.21.15.191:0/318552803 learned_addr learned my addr 172.21.15.191:0/318552803 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.171+0000 7f3d4abb8700 1 -- 172.21.15.191:0/318552803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716ab60 con 0x56511759b400 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.171+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/318552803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5651175cc000 tx=0x565117233980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4ec925d3b660ca server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.171+0000 7f3d31b86700 1 -- 172.21.15.191:0/318552803 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651171fb760 con 0x56511759b400 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.171+0000 7f3d31b86700 1 -- 172.21.15.191:0/318552803 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651175d2000 con 0x56511759b400 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.171+0000 7f3d31b86700 1 -- 172.21.15.191:0/318552803 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651175d4000 con 0x56511759b400 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.172+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/318552803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 msgr2=0x56511742c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.172+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/318552803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5651175cc000 tx=0x565117233980 comp rx=0 tx=0).stop 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/318552803 shutdown_connections 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/318552803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/318552803 >> 172.21.15.191:0/318552803 conn(0x56511759b000 msgr2=0x565117230000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/318552803 shutdown_connections 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/318552803 wait complete. 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 Processor -- start 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.174+0000 7f3d4d40fa80 1 -- start start 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.175+0000 7f3d4d40fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.175+0000 7f3d4d40fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651171e4000 con 0x56511759b400 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.175+0000 7f3d4abb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.175+0000 7f3d4abb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46018/0 (socket says 172.21.15.191:46018) 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.175+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 learned_addr learned my addr 172.21.15.191:0/1698429208 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.176+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511716ad00 con 0x56511759b400 2024-04-27T04:35:05.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.176+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5651175cc2a0 tx=0x5651172338c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.178+0000 7f3d30383700 1 -- 172.21.15.191:0/1698429208 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651175d4160 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.178+0000 7f3d30383700 1 -- 172.21.15.191:0/1698429208 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651175d2180 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.178+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56511716a340 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.178+0000 7f3d30383700 1 -- 172.21.15.191:0/1698429208 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5651175d42c0 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.178+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56511716bd40 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.179+0000 7f3d30383700 1 -- 172.21.15.191:0/1698429208 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565117ee5b80 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.179+0000 7f3d30383700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565117650000 0x56511742d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.179+0000 7f3d30383700 1 -- 172.21.15.191:0/1698429208 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6264+0+0 (secure 0 0 0) 0x5651171bd500 con 0x56511759b400 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.179+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56511759a000 0x565117666000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.179+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565117650000 0x56511742d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.179+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759b800 con 0x56511759a000 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.180+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56511759a000 0x565117666000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.180+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56511759a000 0x565117666000 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.180+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565117650000 0x56511742d700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x565117533c80 tx=0x5651175d5320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.181+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v32'55 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.181+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56511759a800 0x565117666580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.181+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650400 con 0x56511759a800 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.181+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56511759a800 0x565117666580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.182+0000 7f3d4abb8700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56511759a800 0x565117666580 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.183+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.183+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565117650800 0x565117666b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.183+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x565117650800 2024-04-27T04:35:05.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.183+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565117650800 0x565117666b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.183+0000 7f3d4bbba700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565117650800 0x565117666b00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.184+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.184+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x56511759a000 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.185+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v32'175 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.185+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565117651400 0x565117667080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.185+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x565117651400 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.185+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565117651400 0x565117667080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.185+0000 7f3d4b3b9700 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565117651400 0x565117667080 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.186+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v32'65 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.186+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651c00 con 0x565117650800 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.187+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v32'50 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.187+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x56511759a000 2024-04-27T04:35:05.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.188+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v32'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.188+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759a400 con 0x56511759a800 2024-04-27T04:35:05.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.189+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v32'45 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.189+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759ac00 con 0x565117651400 2024-04-27T04:35:05.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.190+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v32'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.190+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759b800 con 0x565117650800 2024-04-27T04:35:05.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.191+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.191+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x565117651400 2024-04-27T04:35:05.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.192+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v32'66 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.192+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e400 con 0x56511759a800 2024-04-27T04:35:05.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.193+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v32'26 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.193+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x565117651400 2024-04-27T04:35:05.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.193+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.193+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769ec00 con 0x56511759a800 2024-04-27T04:35:05.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.194+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v32'59 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.194+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x56511759a000 2024-04-27T04:35:05.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.195+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v32'176 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.195+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f400 con 0x565117650800 2024-04-27T04:35:05.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.196+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v32'56 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.196+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f800 con 0x565117651400 2024-04-27T04:35:05.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.197+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v32'67 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.197+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x565117650800 2024-04-27T04:35:05.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.198+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v32'34 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.198+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x56511759a000 2024-04-27T04:35:05.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.199+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v32'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.199+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x56511759a000 2024-04-27T04:35:05.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.200+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v32'177 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.200+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e400 con 0x565117650800 2024-04-27T04:35:05.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.201+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v32'57 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.201+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x565117651400 2024-04-27T04:35:05.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.202+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v32'38 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.202+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759ac00 con 0x565117650800 2024-04-27T04:35:05.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.203+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v32'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.203+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759a400 con 0x565117650800 2024-04-27T04:35:05.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.204+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v32'40 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.204+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x565117650800 2024-04-27T04:35:05.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.205+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v32'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.205+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x565117650800 2024-04-27T04:35:05.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.206+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v32'51 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.206+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650400 con 0x56511759a800 2024-04-27T04:35:05.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.207+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v32'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.207+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117143c00 con 0x565117650800 2024-04-27T04:35:05.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.207+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v32'34 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.208+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f400 con 0x56511759a800 2024-04-27T04:35:05.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.209+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v32'46 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.209+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x565117650800 2024-04-27T04:35:05.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.210+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v32'58 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.210+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e400 con 0x56511759a800 2024-04-27T04:35:05.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.210+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v32'60 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.210+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759bc00 con 0x565117651400 2024-04-27T04:35:05.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.211+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.212+0000 7f3d2db7e700 20 reqs_thread_entry: start 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.212+0000 7f3d4d40fa80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.212+0000 7f3d4d40fa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.212+0000 7f3d2bb7a700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:35 2024 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.212+0000 7f3d2b358700 20 reqs_thread_entry: start 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x56511759a000 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651c00 con 0x56511759a000 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x56511759a000 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.213+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759bc00 con 0x56511759a800 2024-04-27T04:35:05.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f800 con 0x56511759a800 2024-04-27T04:35:05.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x56511759a800 2024-04-27T04:35:05.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x565117650800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.214+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x565117650800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759ac00 con 0x565117650800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759a400 con 0x56511759a000 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv168 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759bc00 con 0x56511759a000 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv168 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.215+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x56511759a000 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv168 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x565117651400 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x565117651400 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv60 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769ec00 con 0x565117651400 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.216+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f400 con 0x565117650800 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e400 con 0x565117650800 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv47 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x565117650800 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x56511759a000 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.217+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650400 con 0x56511759a000 2024-04-27T04:35:05.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.218+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.218+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117143c00 con 0x56511759a000 2024-04-27T04:35:05.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.218+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.218+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x56511759a800 2024-04-27T04:35:05.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.218+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.218+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x56511759a800 2024-04-27T04:35:05.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv42 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x56511759a800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759a400 con 0x565117651400 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759bc00 con 0x565117651400 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.219+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651c00 con 0x565117651400 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x565117650800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e000 con 0x565117650800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e400 con 0x565117650800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.220+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e800 con 0x565117651400 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.221+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.221+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770ec00 con 0x565117651400 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.221+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.221+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f000 con 0x565117651400 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.221+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.221+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f400 con 0x56511759a800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f800 con 0x56511759a800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770fc00 con 0x56511759a800 2024-04-27T04:35:05.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f400 con 0x565117651400 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.222+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e800 con 0x565117651400 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.223+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.223+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x565117651400 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.223+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.223+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650400 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.223+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.223+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651c00 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.224+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.224+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759a400 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.224+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.224+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759bc00 con 0x56511759a000 2024-04-27T04:35:05.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.224+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv170 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.224+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117143c00 con 0x56511759a000 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv170 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f800 con 0x56511759a000 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv170 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x565117650800 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x565117650800 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x565117650800 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.225+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x565117651400 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv64 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769ec00 con 0x565117651400 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv64 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f800 con 0x565117651400 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv64 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x565117650800 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.226+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f400 con 0x565117650800 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e400 con 0x565117650800 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x56511759a000 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.227+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770ec00 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f000 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv172 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f400 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv172 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e800 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv172 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x56511759a000 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e000 con 0x565117650800 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.228+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e400 con 0x565117650800 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 29 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 30 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x565117651400 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x565117651400 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759bc00 con 0x565117651400 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.229+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759ac00 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 31 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117143c00 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 32 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x565117650800 2024-04-27T04:35:05.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 33 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 34 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.230+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770ec00 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 35 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f000 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 36 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f400 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 37 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e800 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 38 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769ec00 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 39 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.231+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f800 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 40 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x565117650800 2024-04-27T04:35:05.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 41 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769fc00 con 0x565117650800 2024-04-27T04:35:05.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 42 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x56511759a800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.232+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x56511759a800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117143c00 con 0x56511759a800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759ac00 con 0x565117650800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 43 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e000 con 0x565117650800 2024-04-27T04:35:05.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 44 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.233+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e400 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 45 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f000 con 0x56511759a800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e000 con 0x56511759a800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f400 con 0x56511759a800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.234+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e400 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 46 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650c00 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 47 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117650400 con 0x565117650800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4bbba700 1 -- 172.21.15.191:0/1698429208 <== osd.3 v2:172.21.15.191:6800/41887 48 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117582b40 con 0x565117650800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651c00 con 0x56511759a800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.235+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651000 con 0x56511759a800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x565117651800 con 0x56511759a800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4abb8700 1 -- 172.21.15.191:0/1698429208 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583440 con 0x56511759a800 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759ac00 con 0x565117651400 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511759a400 con 0x565117651400 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 27 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770ec00 con 0x565117651400 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.236+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 <== osd.2 v2:172.21.15.191:6816/41888 28 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x565117583200 con 0x565117651400 2024-04-27T04:35:05.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.237+0000 7f3d32387700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.237+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94906284146688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56511770f000 con 0x565117143800 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.237+0000 7f3d443ab700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_map(32..32 src has 1..32) v4 ==== 491+0+0 (crc 0 0 0) 0x5651171bc8c0 con 0x565117143800 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.238+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94906284146688] v32'178 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.238+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94906282547968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f400 con 0x5651174d9400 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.239+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94906282547968] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.239+0000 7f3d4d40fa80 20 remove_watcher() i=0 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.239+0000 7f3d4d40fa80 2 removed watcher, disabling cache 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.239+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94906282549120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e800 con 0x565117143800 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.240+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94906282549120] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.240+0000 7f3d4d40fa80 20 remove_watcher() i=1 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.240+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94906282550272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770e000 con 0x5651174d9c00 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.241+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/2462756345 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94906282550272] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583200 con 0x5651174d9c00 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.241+0000 7f3d4d40fa80 20 remove_watcher() i=2 2024-04-27T04:35:05.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.241+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94906282551424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770f800 con 0x5651174d9400 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.242+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94906282551424] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117583440 con 0x5651174d9400 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.242+0000 7f3d4d40fa80 20 remove_watcher() i=3 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.242+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94906282552576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511770fc00 con 0x565117143800 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.243+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94906282552576] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x565117143800 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.243+0000 7f3d4d40fa80 20 remove_watcher() i=4 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.243+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94906282553728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769ec00 con 0x5651174d9000 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.244+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94906282553728] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.244+0000 7f3d4d40fa80 20 remove_watcher() i=5 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.244+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94906282554880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769f800 con 0x5651174d9000 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.245+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94906282554880] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.245+0000 7f3d4d40fa80 20 remove_watcher() i=6 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.245+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94906282556032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) v8 -- 0x56511769e800 con 0x5651174d9000 2024-04-27T04:35:05.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4bbba700 1 -- 172.21.15.191:0/2462756345 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94906282556032] v32'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565117582b40 con 0x5651174d9000 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 20 remove_watcher() i=7 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5651174d9000 msgr2=0x56511742d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5651174d9000 0x56511742d180 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565117143800 msgr2=0x56511742bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565117143800 0x56511742bb80 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5651174d9400 msgr2=0x56511742cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5651174d9400 0x56511742cc00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5651174d9c00 msgr2=0x56511742c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.246+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5651174d9c00 0x56511742c100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8c00 msgr2=0x56511742b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8c00 0x56511742b600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x565117425890 tx=0x565117233800 comp rx=0 tx=0).stop 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 msgr2=0x56511742ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/2462756345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117143400 0x56511742ab00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x565117fc2930 tx=0x565117490840 comp rx=0 tx=0).stop 2024-04-27T04:35:05.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 reap_dead start 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4abb8700 1 -- 172.21.15.191:0/2462756345 reap_dead start 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 shutdown_connections 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 >> 172.21.15.191:0/2462756345 conn(0x565117142c00 msgr2=0x56511722f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 shutdown_connections 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/2462756345 wait complete. 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56511759a800 msgr2=0x565117666580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56511759a800 0x565117666580 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56511759a000 msgr2=0x565117666000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56511759a000 0x565117666000 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565117651400 msgr2=0x565117667080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x565117651400 0x565117667080 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565117650800 msgr2=0x565117666b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x565117650800 0x565117666b00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565117650000 msgr2=0x56511742d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565117650000 0x56511742d700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x565117533c80 tx=0x5651175d5320 comp rx=0 tx=0).stop 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 msgr2=0x56511742c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/1698429208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56511759b400 0x56511742c680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5651175cc2a0 tx=0x5651172338c0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4b3b9700 1 -- 172.21.15.191:0/1698429208 reap_dead start 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 shutdown_connections 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 >> 172.21.15.191:0/1698429208 conn(0x56511759b000 msgr2=0x565117230000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 shutdown_connections 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.247+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/1698429208 wait complete. 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8000 msgr2=0x56511742b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8000 0x56511742b080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x565117fc2fc0 tx=0x565117490240 comp rx=0 tx=0).stop 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 msgr2=0x56511742a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x565117f1ba40 tx=0x565117233a40 comp rx=0 tx=0).stop 2024-04-27T04:35:05.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 shutdown_connections 2024-04-27T04:35:05.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5651174d8000 0x56511742b080 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 --2- 172.21.15.191:0/922301661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565117142800 0x56511742a580 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 >> 172.21.15.191:0/922301661 conn(0x565117142400 msgr2=0x56511722f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 shutdown_connections 2024-04-27T04:35:05.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.248+0000 7f3d4d40fa80 1 -- 172.21.15.191:0/922301661 wait complete. 2024-04-27T04:35:05.260 INFO:teuthology.orchestra.run.smithi191.stdout:[] 2024-04-27T04:35:05.260 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T04:35:05.260 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T04:35:05.260 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T04:35:05.260 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T04:35:05.305 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:05.306 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15bc031a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15bc031a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56141e503500 con 0x56141e1d8c00 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15bc031a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x56141e27aa00 con 0x56141e1d8800 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15b9fdb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15b97da700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x56141e1d8c00 0x56141e2c4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:33554/0 (socket says 172.21.15.191:33554) 2024-04-27T04:35:05.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.305+0000 7f15b97da700 1 -- 172.21.15.191:0/791613728 learned_addr learned my addr 172.21.15.191:0/791613728 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15b9fdb700 1 -- 172.21.15.191:0/791613728 >> v1:172.21.15.138:6789/0 conn(0x56141e1d8c00 legacy=0x56141e2c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15b9fdb700 1 -- 172.21.15.191:0/791613728 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e201d40 con 0x56141e1d8800 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/791613728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x56141e4bb9e0 tx=0x56141e2c9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dcade66acb0468 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15b8fd9700 1 -- 172.21.15.191:0/791613728 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e2914a0 con 0x56141e1d8800 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15b8fd9700 1 -- 172.21.15.191:0/791613728 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e503800 con 0x56141e1d8800 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15b8fd9700 1 -- 172.21.15.191:0/791613728 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e291600 con 0x56141e1d8800 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 -- 172.21.15.191:0/791613728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 msgr2=0x56141e4c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 --2- 172.21.15.191:0/791613728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x56141e4bb9e0 tx=0x56141e2c9920 comp rx=0 tx=0).stop 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 -- 172.21.15.191:0/791613728 shutdown_connections 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 --2- 172.21.15.191:0/791613728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 -- 172.21.15.191:0/791613728 >> 172.21.15.191:0/791613728 conn(0x56141e1d8400 msgr2=0x56141e2c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 -- 172.21.15.191:0/791613728 shutdown_connections 2024-04-27T04:35:05.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.306+0000 7f15bc031a80 1 -- 172.21.15.191:0/791613728 wait complete. 2024-04-27T04:35:05.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.307+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:05.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.307+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.307+0000 7f15bc031a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.307+0000 7f15bc031a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e27a500 con 0x56141e1d8800 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b9fdb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b9fdb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46026/0 (socket says 172.21.15.191:46026) 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1428874554 learned_addr learned my addr 172.21.15.191:0/1428874554 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1428874554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e201ba0 con 0x56141e1d8800 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/1428874554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x56141efb00c0 tx=0x56141e2c9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49702d92a1c075c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b87d8700 1 -- 172.21.15.191:0/1428874554 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e291760 con 0x56141e1d8800 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b87d8700 1 -- 172.21.15.191:0/1428874554 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e503b00 con 0x56141e1d8800 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15b87d8700 1 -- 172.21.15.191:0/1428874554 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e2911e0 con 0x56141e1d8800 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15bc031a80 1 -- 172.21.15.191:0/1428874554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 msgr2=0x56141e4c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.308+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1428874554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x56141efb00c0 tx=0x56141e2c9800 comp rx=0 tx=0).stop 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 -- 172.21.15.191:0/1428874554 shutdown_connections 2024-04-27T04:35:05.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1428874554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 -- 172.21.15.191:0/1428874554 >> 172.21.15.191:0/1428874554 conn(0x56141e1d8400 msgr2=0x56141e2c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 -- 172.21.15.191:0/1428874554 shutdown_connections 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 -- 172.21.15.191:0/1428874554 wait complete. 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15bc031a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e27a3c0 con 0x56141e1d8800 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15b9fdb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15b9fdb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46042/0 (socket says 172.21.15.191:46042) 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.309+0000 7f15b9fdb700 1 -- 172.21.15.191:0/2458490171 learned_addr learned my addr 172.21.15.191:0/2458490171 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15b9fdb700 1 -- 172.21.15.191:0/2458490171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e201a00 con 0x56141e1d8800 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x56141efb1920 tx=0x56141e2c9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15b6fd5700 1 -- 172.21.15.191:0/2458490171 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e2914a0 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15b6fd5700 1 -- 172.21.15.191:0/2458490171 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e503c80 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15b6fd5700 1 -- 172.21.15.191:0/2458490171 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e291600 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56141e201860 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.310+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56141e2016c0 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b6fd5700 1 -- 172.21.15.191:0/2458490171 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56141ef7ab00 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15bc031a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15bc031a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e214640 con 0x56141e1d9400 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b6fd5700 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566000 0x56141e4c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15ba7dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b6fd5700 1 -- 172.21.15.191:0/2458490171 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6264+0+0 (secure 0 0 0) 0x56141e253180 con 0x56141e1d8800 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15ba7dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46058/0 (socket says 172.21.15.191:46058) 2024-04-27T04:35:05.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3231879930 learned_addr learned my addr 172.21.15.191:0/3231879930 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566000 0x56141e4c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3231879930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e200000 con 0x56141e1d9400 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566000 0x56141e4c1080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x56141f06aed0 tx=0x56141e52ec60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/3231879930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x56141efb1fb0 tx=0x56141e52eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894b1e4d203a81a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b47d0700 1 -- 172.21.15.191:0/3231879930 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e2914a0 con 0x56141e1d9400 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b47d0700 1 -- 172.21.15.191:0/3231879930 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e279c80 con 0x56141e1d9400 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.311+0000 7f15b47d0700 1 -- 172.21.15.191:0/3231879930 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e291600 con 0x56141e1d9400 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- 172.21.15.191:0/3231879930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 msgr2=0x56141e4c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3231879930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x56141efb1fb0 tx=0x56141e52eb40 comp rx=0 tx=0).stop 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- 172.21.15.191:0/3231879930 shutdown_connections 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3231879930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- 172.21.15.191:0/3231879930 >> 172.21.15.191:0/3231879930 conn(0x56141e1d8c00 msgr2=0x56141e2c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- 172.21.15.191:0/3231879930 shutdown_connections 2024-04-27T04:35:05.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- 172.21.15.191:0/3231879930 wait complete. 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.312+0000 7f15bc031a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e2148c0 con 0x56141e1d9400 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15ba7dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15ba7dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46070/0 (socket says 172.21.15.191:46070) 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 learned_addr learned my addr 172.21.15.191:0/1676948828 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e200ea0 con 0x56141e1d9400 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x56141f06a0f0 tx=0x56141e2c98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15b2fcd700 1 -- 172.21.15.191:0/1676948828 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e291760 con 0x56141e1d9400 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15b2fcd700 1 -- 172.21.15.191:0/1676948828 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e279b00 con 0x56141e1d9400 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56141e200820 con 0x56141e1d9400 2024-04-27T04:35:05.325 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15b2fcd700 1 -- 172.21.15.191:0/1676948828 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e2914a0 con 0x56141e1d9400 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.313+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56141e2009c0 con 0x56141e1d9400 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15b2fcd700 1 -- 172.21.15.191:0/1676948828 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56141ef7adc0 con 0x56141e1d9400 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15b2fcd700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566c00 0x56141e4c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15b2fcd700 1 -- 172.21.15.191:0/1676948828 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6264+0+0 (secure 0 0 0) 0x56141e253340 con 0x56141e1d9400 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15b97da700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566c00 0x56141e4c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e1d9800 0x56141e4c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e1d9c00 con 0x56141e1d9800 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e1d9800 0x56141e4c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15b97da700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566c00 0x56141e4c1600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56141e583ad0 tx=0x56141e6180c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.314+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e1d9800 0x56141e4c1b80 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:05.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 realm 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e567000 con 0x56141e1d9800 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e567400 con 0x56141e1d9800 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e567800 con 0x56141e1d9800 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:05.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e567c00 0x56141e4c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.315+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e567000 con 0x56141e567c00 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e567c00 0x56141e4c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e567c00 0x56141e4c2100 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15bc031a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15bc031a80 20 searching for the correct realm 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e567400 con 0x56141e567c00 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 32 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e567400 0x56141e4c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e567000 con 0x56141e567400 2024-04-27T04:35:05.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.316+0000 7f15b97da700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e567400 0x56141e4c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.317+0000 7f15b97da700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e567400 0x56141e4c2c00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.317+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.317+0000 7f15b97da700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e567000 0x56141e4c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.317+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.317+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e567000 0x56141e4c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.318+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e567000 0x56141e4c3180 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.318+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.318+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.318+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:05.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:05.318+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.186+0000 7f15b2fcd700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_map(33..33 src has 1..33) v4 ==== 312+0+0 (crc 0 0 0) 0x56141e253500 con 0x56141e1d9800 2024-04-27T04:35:06.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.186+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.186+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.187+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.187+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.187+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.187+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567400 2024-04-27T04:35:06.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.187+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.187+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.188+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.188+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.188+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.188+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.189+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.189+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.189+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.189+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567400 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.189+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.189+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.190+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.190+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.190+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.190+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.190+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.190+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.191+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567400 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.192+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567400 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.193+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567400 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.194+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.194+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567c00 2024-04-27T04:35:06.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.194+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.194+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.194+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.194+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567400 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e1d9c00 con 0x56141e1d9800 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.195+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e400 con 0x56141e1d9800 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e800 con 0x56141e567c00 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 zone default found 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 4 Realm: () 2024-04-27T04:35:06.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 zonegroup default 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62ec00 con 0x56141e567000 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f000 con 0x56141e567c00 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.196+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f400 con 0x56141e567c00 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15bc031a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15bc031a80 20 started sync module instance, tier type = 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15bc031a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.197+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e567400 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.198+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.198+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94644407597824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e800 con 0x56141e567400 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.198+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f000 con 0x56141e1d9800 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.199+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94644407597824] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.199+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(46 notify.1 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.199+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94644407598976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f400 con 0x56141e1d9800 2024-04-27T04:35:06.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.199+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e400 con 0x56141e567c00 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.200+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94644407598976] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.200+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.201+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94644407600128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62ec00 con 0x56141e567c00 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.201+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e1d9c00 con 0x56141e567400 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.202+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94644407600128] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.202+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.202+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94644407601280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e567800 con 0x56141e567400 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.202+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e1d9800 2024-04-27T04:35:06.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.203+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(52 notify.4 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.203+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94644407602432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e1d9800 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.203+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e800 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.203+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94644407601280] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.204+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.204+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94644407603584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62fc00 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.204+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f000 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.204+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94644407602432] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.205+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94644407603584] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.205+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.205+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94644407604736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f400 con 0x56141e567000 2024-04-27T04:35:06.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.205+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e000 con 0x56141e567000 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94644407604736] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94644407605888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e567000 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=0 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=1 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=2 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=3 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=4 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=5 2024-04-27T04:35:06.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.206+0000 7f15bc031a80 20 add_watcher() i=6 2024-04-27T04:35:06.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.207+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94644407605888] v33'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e567000 2024-04-27T04:35:06.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.207+0000 7f15bc031a80 20 add_watcher() i=7 2024-04-27T04:35:06.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.207+0000 7f15bc031a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:06.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.207+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e800 con 0x56141e1d9800 2024-04-27T04:35:06.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.208+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.208+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94644409180160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62fc00 con 0x56141e1d9800 2024-04-27T04:35:06.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.209+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94644409180160] v33'179 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56141e61cb40 con 0x56141e1d9800 2024-04-27T04:35:06.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.209+0000 7f15a0fa9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.209+0000 7f15bc031a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.209+0000 7f15bc031a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.209+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.209+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15bc031a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15bc031a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e27a3c0 con 0x56141e62f400 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15b97da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15b97da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46080/0 (socket says 172.21.15.191:46080) 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15b97da700 1 -- 172.21.15.191:0/1967940289 learned_addr learned my addr 172.21.15.191:0/1967940289 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15b97da700 1 -- 172.21.15.191:0/1967940289 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e200b60 con 0x56141e62f400 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15b97da700 1 --2- 172.21.15.191:0/1967940289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x56141e582f00 tx=0x56141e52ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fd6a94dccdd7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.210+0000 7f15a07a8700 1 -- 172.21.15.191:0/1967940289 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e64c000 con 0x56141e62f400 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15a07a8700 1 -- 172.21.15.191:0/1967940289 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e650000 con 0x56141e62f400 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15a07a8700 1 -- 172.21.15.191:0/1967940289 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e64c160 con 0x56141e62f400 2024-04-27T04:35:06.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 -- 172.21.15.191:0/1967940289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 msgr2=0x56141e4c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1967940289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x56141e582f00 tx=0x56141e52ff80 comp rx=0 tx=0).stop 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 -- 172.21.15.191:0/1967940289 shutdown_connections 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1967940289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 -- 172.21.15.191:0/1967940289 >> 172.21.15.191:0/1967940289 conn(0x56141e62f000 msgr2=0x56141e2c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 -- 172.21.15.191:0/1967940289 shutdown_connections 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.211+0000 7f15bc031a80 1 -- 172.21.15.191:0/1967940289 wait complete. 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15bc031a80 1 Processor -- start 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15bc031a80 1 -- start start 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15bc031a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15bc031a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e27a500 con 0x56141e62f400 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15b97da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15b97da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:46096/0 (socket says 172.21.15.191:46096) 2024-04-27T04:35:06.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.212+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 learned_addr learned my addr 172.21.15.191:0/3905785848 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e200d00 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f15b97da700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x56141e64e210 tx=0x56141e6184e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f159efa5700 1 -- 172.21.15.191:0/3905785848 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e64c2c0 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f159efa5700 1 -- 172.21.15.191:0/3905785848 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56141e650180 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f159efa5700 1 -- 172.21.15.191:0/3905785848 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56141e64c000 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56141e200340 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.213+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56141e200b60 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f159efa5700 1 -- 172.21.15.191:0/3905785848 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56141ef7bb80 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f159efa5700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e6d0000 0x56141e4c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f159efa5700 1 -- 172.21.15.191:0/3905785848 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x56141e253340 con 0x56141e62f400 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e6d0000 0x56141e4c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e62e000 0x56141e6e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e62e000 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e62e000 0x56141e6e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e6d0000 0x56141e4c3700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56141e5cd860 tx=0x56141e618360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.214+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e62e000 0x56141e6e6000 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.215+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'56 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e62e000 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.215+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e62e800 0x56141e6e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.216+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d0400 con 0x56141e62e800 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.216+0000 7f15b97da700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e62e800 0x56141e6e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.216+0000 7f15b97da700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e62e800 0x56141e6e6580 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.217+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.217+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e6d0800 0x56141e6e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.217+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d0c00 con 0x56141e6d0800 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.217+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e6d0800 0x56141e6e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.217+0000 7f15ba7dc700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e6d0800 0x56141e6e6b00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.218+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.219+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1000 con 0x56141e62e000 2024-04-27T04:35:06.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.220+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'180 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e62e000 2024-04-27T04:35:06.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.220+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e6d1400 0x56141e6e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.220+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1800 con 0x56141e6d1400 2024-04-27T04:35:06.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.220+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e6d1400 0x56141e6e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.220+0000 7f15b9fdb700 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e6d1400 0x56141e6e7080 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.221+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'68 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.221+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1c00 con 0x56141e6d0800 2024-04-27T04:35:06.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.222+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'52 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.222+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1000 con 0x56141e62e000 2024-04-27T04:35:06.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.223+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e62e000 2024-04-27T04:35:06.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.223+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e400 con 0x56141e62e800 2024-04-27T04:35:06.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.224+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'47 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.224+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62ec00 con 0x56141e6d1400 2024-04-27T04:35:06.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.225+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.225+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e6d0800 2024-04-27T04:35:06.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.226+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.226+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e000 con 0x56141e6d1400 2024-04-27T04:35:06.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.227+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'69 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.227+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e400 con 0x56141e62e800 2024-04-27T04:35:06.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.228+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'28 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.228+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e800 con 0x56141e6d1400 2024-04-27T04:35:06.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.229+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.229+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71ec00 con 0x56141e62e800 2024-04-27T04:35:06.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.231+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'61 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.232 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.231+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71f000 con 0x56141e62e000 2024-04-27T04:35:06.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.232+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'181 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e62e000 2024-04-27T04:35:06.233 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.232+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71f400 con 0x56141e6d0800 2024-04-27T04:35:06.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.233+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'59 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.234 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.233+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71f800 con 0x56141e6d1400 2024-04-27T04:35:06.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.234+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'70 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.235 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.234+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71fc00 con 0x56141e6d0800 2024-04-27T04:35:06.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.235+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'35 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.235+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e000 con 0x56141e62e000 2024-04-27T04:35:06.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.236+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e62e000 2024-04-27T04:35:06.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.236+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e800 con 0x56141e62e000 2024-04-27T04:35:06.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.237+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'182 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e62e000 2024-04-27T04:35:06.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.237+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e400 con 0x56141e6d0800 2024-04-27T04:35:06.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.238+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'60 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.238+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62ec00 con 0x56141e6d1400 2024-04-27T04:35:06.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.239+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'39 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.239+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e400 con 0x56141e6d0800 2024-04-27T04:35:06.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.240+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.240+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1800 con 0x56141e6d0800 2024-04-27T04:35:06.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.241+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'41 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.241+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1000 con 0x56141e6d0800 2024-04-27T04:35:06.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.242+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.242+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d0400 con 0x56141e6d0800 2024-04-27T04:35:06.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.243+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'53 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.243+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e1d9c00 con 0x56141e62e800 2024-04-27T04:35:06.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.244+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.244+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71f400 con 0x56141e6d0800 2024-04-27T04:35:06.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.245+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'35 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.245+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71fc00 con 0x56141e62e800 2024-04-27T04:35:06.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.246+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'48 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.246+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e400 con 0x56141e6d0800 2024-04-27T04:35:06.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.247+0000 7f15ba7dc700 1 -- 172.21.15.191:0/3905785848 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'61 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e6d0800 2024-04-27T04:35:06.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.247+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62fc00 con 0x56141e62e800 2024-04-27T04:35:06.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.249+0000 7f15b97da700 1 -- 172.21.15.191:0/3905785848 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'62 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e62e800 2024-04-27T04:35:06.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.249+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e6d1400 2024-04-27T04:35:06.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.250+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e6d1400 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f159c7a0700 20 reqs_thread_entry: start 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f15bc031a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f15bc031a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f159a79c700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:36 2024 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f1599f7a700 20 reqs_thread_entry: start 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f15bc031a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f15bc031a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.251+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e6d1c00 con 0x56141e1d9800 2024-04-27T04:35:06.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x56141e61cd80 con 0x56141e1d9800 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15bc031a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15bc031a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15bc031a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15bc031a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62ec00 con 0x56141e567400 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.252+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv27 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.255 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T04:35:06.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.253+0000 7f15a0fa9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.253+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94644409180160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e1d9800 2024-04-27T04:35:06.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.254+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94644409180160] v33'183 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e1d9800 2024-04-27T04:35:06.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.254+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94644407597824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71f000 con 0x56141e567400 2024-04-27T04:35:06.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.255+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94644407597824] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.255+0000 7f15bc031a80 20 remove_watcher() i=0 2024-04-27T04:35:06.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.255+0000 7f15bc031a80 2 removed watcher, disabling cache 2024-04-27T04:35:06.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.255+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94644407598976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71f800 con 0x56141e1d9800 2024-04-27T04:35:06.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.256+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94644407598976] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e1d9800 2024-04-27T04:35:06.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.256+0000 7f15bc031a80 20 remove_watcher() i=1 2024-04-27T04:35:06.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.256+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94644407600128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e000 con 0x56141e567c00 2024-04-27T04:35:06.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.257+0000 7f15b9fdb700 1 -- 172.21.15.191:0/1676948828 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94644407600128] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d200 con 0x56141e567c00 2024-04-27T04:35:06.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.257+0000 7f15bc031a80 20 remove_watcher() i=2 2024-04-27T04:35:06.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.257+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94644407601280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e71e800 con 0x56141e567400 2024-04-27T04:35:06.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.259+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94644407601280] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61d440 con 0x56141e567400 2024-04-27T04:35:06.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.259+0000 7f15bc031a80 20 remove_watcher() i=3 2024-04-27T04:35:06.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.259+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94644407602432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62e400 con 0x56141e1d9800 2024-04-27T04:35:06.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.260+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94644407602432] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e1d9800 2024-04-27T04:35:06.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.260+0000 7f15bc031a80 20 remove_watcher() i=4 2024-04-27T04:35:06.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.260+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94644407603584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62f800 con 0x56141e567000 2024-04-27T04:35:06.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.261+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94644407603584] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e567000 2024-04-27T04:35:06.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.261+0000 7f15bc031a80 20 remove_watcher() i=5 2024-04-27T04:35:06.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.261+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94644407604736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62fc00 con 0x56141e567000 2024-04-27T04:35:06.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.262+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94644407604736] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e567000 2024-04-27T04:35:06.263 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.262+0000 7f15bc031a80 20 remove_watcher() i=6 2024-04-27T04:35:06.264 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.262+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94644407605888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56141e62ec00 con 0x56141e567000 2024-04-27T04:35:06.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15ba7dc700 1 -- 172.21.15.191:0/1676948828 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94644407605888] v33'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56141e61cd80 con 0x56141e567000 2024-04-27T04:35:06.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 20 remove_watcher() i=7 2024-04-27T04:35:06.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e567000 msgr2=0x56141e4c3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e567000 0x56141e4c3180 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e1d9800 msgr2=0x56141e4c1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e1d9800 0x56141e4c1b80 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e567400 msgr2=0x56141e4c2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e567400 0x56141e4c2c00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e567c00 msgr2=0x56141e4c2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e567c00 0x56141e4c2100 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566c00 msgr2=0x56141e4c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566c00 0x56141e4c1600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56141e583ad0 tx=0x56141e6180c0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 msgr2=0x56141e4c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 --2- 172.21.15.191:0/1676948828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d9400 0x56141e4c0b00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x56141f06a0f0 tx=0x56141e2c98c0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 reap_dead start 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15b97da700 1 -- 172.21.15.191:0/1676948828 reap_dead start 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 shutdown_connections 2024-04-27T04:35:06.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 >> 172.21.15.191:0/1676948828 conn(0x56141e1d8c00 msgr2=0x56141e2c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 shutdown_connections 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.263+0000 7f15bc031a80 1 -- 172.21.15.191:0/1676948828 wait complete. 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e62e800 msgr2=0x56141e6e6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56141e62e800 0x56141e6e6580 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e62e000 msgr2=0x56141e6e6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x56141e62e000 0x56141e6e6000 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e6d1400 msgr2=0x56141e6e7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56141e6d1400 0x56141e6e7080 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e6d0800 msgr2=0x56141e6e6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56141e6d0800 0x56141e6e6b00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e6d0000 msgr2=0x56141e4c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e6d0000 0x56141e4c3700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56141e5cd860 tx=0x56141e618360 comp rx=0 tx=0).stop 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 msgr2=0x56141e4c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x56141e64e210 tx=0x56141e6184e0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15b9fdb700 1 -- 172.21.15.191:0/3905785848 reap_dead start 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 shutdown_connections 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/3905785848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e62f400 0x56141e4c2680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 >> 172.21.15.191:0/3905785848 conn(0x56141e62f000 msgr2=0x56141e2c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 shutdown_connections 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/3905785848 wait complete. 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566000 msgr2=0x56141e4c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566000 0x56141e4c1080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x56141f06aed0 tx=0x56141e52ec60 comp rx=0 tx=0).stop 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 msgr2=0x56141e4c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x56141efb1920 tx=0x56141e2c9920 comp rx=0 tx=0).stop 2024-04-27T04:35:06.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 shutdown_connections 2024-04-27T04:35:06.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56141e566000 0x56141e4c1080 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 --2- 172.21.15.191:0/2458490171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56141e1d8800 0x56141e4c0580 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 >> 172.21.15.191:0/2458490171 conn(0x56141e1d8400 msgr2=0x56141e2c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 shutdown_connections 2024-04-27T04:35:06.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.264+0000 7f15bc031a80 1 -- 172.21.15.191:0/2458490171 wait complete. 2024-04-27T04:35:06.272 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T04:35:06.280 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-04-27T04:35:06.285 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:06.317 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:06.331 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:06.375 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-27T04:35:06.379 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:06.379 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T04:35:06.379 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T04:35:06.380 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-27T04:35:06.426 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:06.426 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:06.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.425+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.425+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.425+0000 7fabd7d89a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.425+0000 7fabd7d89a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cff4d59500 con 0x55cff4a2ec00 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.425+0000 7fabd7d89a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cff4ad0a00 con 0x55cff4a2e800 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd5d33700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd5532700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55cff4a2ec00 0x55cff4b1a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57104/0 (socket says 172.21.15.191:57104) 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd5532700 1 -- 172.21.15.191:0/3132206982 learned_addr learned my addr 172.21.15.191:0/3132206982 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd5d33700 1 -- 172.21.15.191:0/3132206982 >> v1:172.21.15.138:6789/0 conn(0x55cff4a2ec00 legacy=0x55cff4b1a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd5d33700 1 -- 172.21.15.191:0/3132206982 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a57d40 con 0x55cff4a2e800 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd5d33700 1 --2- 172.21.15.191:0/3132206982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55cff4d119e0 tx=0x55cff4b1f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ea59f34fc6f47c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd4d31700 1 -- 172.21.15.191:0/3132206982 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae74a0 con 0x55cff4a2e800 2024-04-27T04:35:06.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd4d31700 1 -- 172.21.15.191:0/3132206982 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4d59800 con 0x55cff4a2e800 2024-04-27T04:35:06.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.426+0000 7fabd4d31700 1 -- 172.21.15.191:0/3132206982 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae7600 con 0x55cff4a2e800 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3132206982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 msgr2=0x55cff4d16580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3132206982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55cff4d119e0 tx=0x55cff4b1f920 comp rx=0 tx=0).stop 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3132206982 shutdown_connections 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3132206982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3132206982 >> 172.21.15.191:0/3132206982 conn(0x55cff4a2e400 msgr2=0x55cff4b1a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3132206982 shutdown_connections 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3132206982 wait complete. 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.427+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd7d89a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd7d89a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff4ad0500 con 0x55cff4a2e800 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd5d33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd5d33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54360/0 (socket says 172.21.15.191:54360) 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd5d33700 1 -- 172.21.15.191:0/1743197271 learned_addr learned my addr 172.21.15.191:0/1743197271 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd5d33700 1 -- 172.21.15.191:0/1743197271 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a57ba0 con 0x55cff4a2e800 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd5d33700 1 --2- 172.21.15.191:0/1743197271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55cff4d10b70 tx=0x55cff4b1f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e38a11ad6d01584 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd4530700 1 -- 172.21.15.191:0/1743197271 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae7760 con 0x55cff4a2e800 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd4530700 1 -- 172.21.15.191:0/1743197271 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4d59b00 con 0x55cff4a2e800 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd4530700 1 -- 172.21.15.191:0/1743197271 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae74a0 con 0x55cff4a2e800 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1743197271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 msgr2=0x55cff4d16580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/1743197271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55cff4d10b70 tx=0x55cff4b1f800 comp rx=0 tx=0).stop 2024-04-27T04:35:06.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.428+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1743197271 shutdown_connections 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/1743197271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1743197271 >> 172.21.15.191:0/1743197271 conn(0x55cff4a2e400 msgr2=0x55cff4b1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1743197271 shutdown_connections 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1743197271 wait complete. 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd7d89a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff4ad03c0 con 0x55cff4a2e800 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd5d33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd5d33700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54368/0 (socket says 172.21.15.191:54368) 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd5d33700 1 -- 172.21.15.191:0/1772599675 learned_addr learned my addr 172.21.15.191:0/1772599675 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd5d33700 1 -- 172.21.15.191:0/1772599675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a57a00 con 0x55cff4a2e800 2024-04-27T04:35:06.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.429+0000 7fabd5d33700 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55cff4d11f80 tx=0x55cff4b1f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd2d2d700 1 -- 172.21.15.191:0/1772599675 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae7600 con 0x55cff4a2e800 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd2d2d700 1 -- 172.21.15.191:0/1772599675 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4d59c80 con 0x55cff4a2e800 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cff4a57860 con 0x55cff4a2e800 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd2d2d700 1 -- 172.21.15.191:0/1772599675 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae78c0 con 0x55cff4a2e800 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cff4a576c0 con 0x55cff4a2e800 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd2d2d700 1 -- 172.21.15.191:0/1772599675 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cff57d0b00 con 0x55cff4a2e800 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd7d89a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd7d89a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff4a6a640 con 0x55cff4a2f400 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.430+0000 7fabd2d2d700 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8000 0x55cff4d17080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd2d2d700 1 -- 172.21.15.191:0/1772599675 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55cff4aa9180 con 0x55cff4a2e800 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd6534700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd6534700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54384/0 (socket says 172.21.15.191:54384) 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd6534700 1 -- 172.21.15.191:0/4236366063 learned_addr learned my addr 172.21.15.191:0/4236366063 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd5d33700 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8000 0x55cff4d17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd6534700 1 -- 172.21.15.191:0/4236366063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a56000 con 0x55cff4a2f400 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd5d33700 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8000 0x55cff4d17080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55cff58baf90 tx=0x55cff4d7cb40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd6534700 1 --2- 172.21.15.191:0/4236366063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55cff58ba7b0 tx=0x55cff4d7ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712812e537426182 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd0528700 1 -- 172.21.15.191:0/4236366063 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae7600 con 0x55cff4a2f400 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd0528700 1 -- 172.21.15.191:0/4236366063 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4acfc80 con 0x55cff4a2f400 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd0528700 1 -- 172.21.15.191:0/4236366063 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae78c0 con 0x55cff4a2f400 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd7d89a80 1 -- 172.21.15.191:0/4236366063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 msgr2=0x55cff4d16b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.431+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/4236366063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55cff58ba7b0 tx=0x55cff4d7ce40 comp rx=0 tx=0).stop 2024-04-27T04:35:06.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 -- 172.21.15.191:0/4236366063 shutdown_connections 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/4236366063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 -- 172.21.15.191:0/4236366063 >> 172.21.15.191:0/4236366063 conn(0x55cff4a2ec00 msgr2=0x55cff4b1b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 -- 172.21.15.191:0/4236366063 shutdown_connections 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 -- 172.21.15.191:0/4236366063 wait complete. 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd7d89a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff4a6a8c0 con 0x55cff4a2f400 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd6534700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd6534700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54394/0 (socket says 172.21.15.191:54394) 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 learned_addr learned my addr 172.21.15.191:0/983566384 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.432+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a56ea0 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabd6534700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55cff58ba690 tx=0x55cff4d7ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabced25700 1 -- 172.21.15.191:0/983566384 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae7760 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabced25700 1 -- 172.21.15.191:0/983566384 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4acfb00 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cff4a56820 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabced25700 1 -- 172.21.15.191:0/983566384 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ae7600 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cff4a569c0 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabced25700 1 -- 172.21.15.191:0/983566384 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cff57d0dc0 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabced25700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8c00 0x55cff4d17600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.433+0000 7fabced25700 1 -- 172.21.15.191:0/983566384 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55cff4aa9340 con 0x55cff4a2f400 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd5532700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8c00 0x55cff4d17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd5532700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8c00 0x55cff4d17600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55cff57fe030 tx=0x55cff4e660c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cff4a2f800 0x55cff4d17b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4a2fc00 con 0x55cff4a2f800 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd6534700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cff4a2f800 0x55cff4d17b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.434+0000 7fabd6534700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cff4a2f800 0x55cff4d17b80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 realm 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9000 con 0x55cff4a2f800 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9400 con 0x55cff4a2f800 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9800 con 0x55cff4a2f800 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.435+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cff4db9c00 0x55cff4d18100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9000 con 0x55cff4db9c00 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd5d33700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cff4db9c00 0x55cff4d18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd5d33700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cff4db9c00 0x55cff4d18100 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd7d89a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd7d89a80 20 searching for the correct realm 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.436+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9400 con 0x55cff4db9c00 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5d33700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cff4db9400 0x55cff4d18c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9000 con 0x55cff4db9400 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5532700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cff4db9400 0x55cff4d18c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5532700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cff4db9400 0x55cff4d18c00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5532700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cff4db9000 0x55cff4d19180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.437+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.438+0000 7fabd6534700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cff4db9000 0x55cff4d19180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.438+0000 7fabd6534700 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cff4db9000 0x55cff4d19180 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.438+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.438+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.438+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.438+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9400 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.439+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9400 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.440+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.441+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9400 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.442+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9400 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.443+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9400 2024-04-27T04:35:06.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9c00 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9400 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.444+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:06.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4a2fc00 con 0x55cff4a2f800 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80400 con 0x55cff4a2f800 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80800 con 0x55cff4db9c00 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 zone default found 2024-04-27T04:35:06.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 4 Realm: () 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 zonegroup default 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.445+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80c00 con 0x55cff4db9000 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81000 con 0x55cff4db9c00 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81400 con 0x55cff4db9c00 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 started sync module instance, tier type = 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.446+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81800 con 0x55cff4db9400 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.447+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.447+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94351653919488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80800 con 0x55cff4db9400 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.447+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81000 con 0x55cff4a2f800 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.448+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94351653919488] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.448+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.448+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94351653920640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81400 con 0x55cff4a2f800 2024-04-27T04:35:06.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.448+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80400 con 0x55cff4db9c00 2024-04-27T04:35:06.478 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.449+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94351653920640] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.450+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.450+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94351653921792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80c00 con 0x55cff4db9c00 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.450+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4a2fc00 con 0x55cff4db9400 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.451+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.451+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94351653921792] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.451+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94351653922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4db9800 con 0x55cff4db9400 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.451+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4a2f800 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.451+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94351653922944] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.452+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.452+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94351653924096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81800 con 0x55cff4a2f800 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.452+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80800 con 0x55cff4db9000 2024-04-27T04:35:06.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.457+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94351653924096] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.457+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.457+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94351653925248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81c00 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.457+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81000 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.458+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94351653925248] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.458+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.459+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94351653926400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81400 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.459+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80000 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.459+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94351653926400] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94351653927552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81800 con 0x55cff4db9000 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=0 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=1 2024-04-27T04:35:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=2 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=3 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=4 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=5 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.460+0000 7fabd7d89a80 20 add_watcher() i=6 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.461+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94351653927552] v33'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.461+0000 7fabd7d89a80 20 add_watcher() i=7 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.461+0000 7fabd7d89a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.461+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e80800 con 0x55cff4a2f800 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.461+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.461+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94351655493632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4e81c00 con 0x55cff4a2f800 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.462+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94351655493632] v33'184 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.462+0000 7fabbcd01700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.462+0000 7fabd7d89a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.462+0000 7fabd7d89a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.462+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd7d89a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd7d89a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff4ad03c0 con 0x55cff4e81400 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd5532700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd5532700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54404/0 (socket says 172.21.15.191:54404) 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd5532700 1 -- 172.21.15.191:0/3331435649 learned_addr learned my addr 172.21.15.191:0/3331435649 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd5532700 1 -- 172.21.15.191:0/3331435649 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a56b60 con 0x55cff4e81400 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabd5532700 1 --2- 172.21.15.191:0/3331435649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55cff4e1f500 tx=0x55cff4d7df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fa6403c98faab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabbc500700 1 -- 172.21.15.191:0/3331435649 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ea2000 con 0x55cff4e81400 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabbc500700 1 -- 172.21.15.191:0/3331435649 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4eaa000 con 0x55cff4e81400 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.463+0000 7fabbc500700 1 -- 172.21.15.191:0/3331435649 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ea2160 con 0x55cff4e81400 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3331435649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 msgr2=0x55cff4d18680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3331435649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55cff4e1f500 tx=0x55cff4d7df80 comp rx=0 tx=0).stop 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3331435649 shutdown_connections 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3331435649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3331435649 >> 172.21.15.191:0/3331435649 conn(0x55cff4e81000 msgr2=0x55cff4b1c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3331435649 shutdown_connections 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3331435649 wait complete. 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 Processor -- start 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.464+0000 7fabd7d89a80 1 -- start start 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd7d89a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd7d89a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff4ad0500 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd5532700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd5532700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54406/0 (socket says 172.21.15.191:54406) 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd5532700 1 -- 172.21.15.191:0/3008694803 learned_addr learned my addr 172.21.15.191:0/3008694803 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd5532700 1 -- 172.21.15.191:0/3008694803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff4a56d00 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd5532700 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55cff4e93200 tx=0x55cff4e664e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabbacfd700 1 -- 172.21.15.191:0/3008694803 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ea22c0 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabbacfd700 1 -- 172.21.15.191:0/3008694803 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cff4eaa180 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabbacfd700 1 -- 172.21.15.191:0/3008694803 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cff4ea2000 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cff4a56340 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.465+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cff4a56b60 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.466+0000 7fabbacfd700 1 -- 172.21.15.191:0/3008694803 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cff57d1b80 con 0x55cff4e81400 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.466+0000 7fabbacfd700 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4f22000 0x55cff4d19700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.466+0000 7fabbacfd700 1 -- 172.21.15.191:0/3008694803 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55cff4aa9500 con 0x55cff4e81400 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.466+0000 7fabd6534700 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4f22000 0x55cff4d19700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.466+0000 7fabd7d89a80 5 note: GC not initialized 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabd6534700 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4f22000 0x55cff4d19700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55cff4d99590 tx=0x55cff4e66360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabb84f8700 20 reqs_thread_entry: start 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabd7d89a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabd7d89a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabb64f4700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:36 2024 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabb5cd2700 20 reqs_thread_entry: start 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabd7d89a80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.467+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f22400 con 0x55cff4db9400 2024-04-27T04:35:06.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 : miss 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.d 8:b151d69a:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f22800 con 0x55cff4db9400 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 info.flags=0x17 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 to cache LRU end 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:06.486 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.468+0000 7fabd7d89a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabd7d89a80 20 get_obj_state: rctx=0x55cff4e577a0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55cff4f921e8 s->prefetch_data=0 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 3.17s0 3:e9a98dfd:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f22c00 con 0x55cff4a2f800 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=848b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 262+0+864 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabd7d89a80 10 manifest: total_size = 5 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabd7d89a80 20 get_obj_state: setting s->obj_tag to _tvMuJJJFvSsvAJmhRSlJgAfRjjW42If 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabbcd01700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.469+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94351655493632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f23000 con 0x55cff4a2f800 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.470+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94351655493632] v33'185 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.470+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94351653919488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f23400 con 0x55cff4db9400 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.471+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94351653919488] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.472+0000 7fabd7d89a80 20 remove_watcher() i=0 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.472+0000 7fabd7d89a80 2 removed watcher, disabling cache 2024-04-27T04:35:06.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.472+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94351653920640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f23c00 con 0x55cff4a2f800 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.473+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94351653920640] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.473+0000 7fabd7d89a80 20 remove_watcher() i=1 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.473+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94351653921792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f23800 con 0x55cff4db9c00 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.474+0000 7fabd5d33700 1 -- 172.21.15.191:0/983566384 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94351653921792] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79200 con 0x55cff4db9c00 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.474+0000 7fabd7d89a80 20 remove_watcher() i=2 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.474+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94351653922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f22400 con 0x55cff4db9400 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.474+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94351653922944] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e79440 con 0x55cff4db9400 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.475+0000 7fabd7d89a80 20 remove_watcher() i=3 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.475+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94351653924096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f22800 con 0x55cff4a2f800 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.476+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94351653924096] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4a2f800 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.476+0000 7fabd7d89a80 20 remove_watcher() i=4 2024-04-27T04:35:06.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.476+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94351653925248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4f23400 con 0x55cff4db9000 2024-04-27T04:35:06.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.477+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94351653925248] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.477+0000 7fabd7d89a80 20 remove_watcher() i=5 2024-04-27T04:35:06.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.477+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94351653926400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4a2fc00 con 0x55cff4db9000 2024-04-27T04:35:06.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.478+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94351653926400] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.478+0000 7fabd7d89a80 20 remove_watcher() i=6 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.478+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94351653927552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cff4a2fc00 con 0x55cff4db9000 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd6534700 1 -- 172.21.15.191:0/983566384 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94351653927552] v33'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cff4e78b40 con 0x55cff4db9000 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 20 remove_watcher() i=7 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cff4db9000 msgr2=0x55cff4d19180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cff4db9000 0x55cff4d19180 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cff4a2f800 msgr2=0x55cff4d17b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cff4a2f800 0x55cff4d17b80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cff4db9400 msgr2=0x55cff4d18c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cff4db9400 0x55cff4d18c00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cff4db9c00 msgr2=0x55cff4d18100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cff4db9c00 0x55cff4d18100 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8c00 msgr2=0x55cff4d17600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8c00 0x55cff4d17600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55cff57fe030 tx=0x55cff4e660c0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 msgr2=0x55cff4d16b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/983566384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2f400 0x55cff4d16b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55cff58ba690 tx=0x55cff4d7ca80 comp rx=0 tx=0).stop 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 reap_dead start 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd5532700 1 -- 172.21.15.191:0/983566384 reap_dead start 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 shutdown_connections 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 >> 172.21.15.191:0/983566384 conn(0x55cff4a2ec00 msgr2=0x55cff4b1b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 shutdown_connections 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/983566384 wait complete. 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4f22000 msgr2=0x55cff4d19700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4f22000 0x55cff4d19700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55cff4d99590 tx=0x55cff4e66360 comp rx=0 tx=0).stop 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 msgr2=0x55cff4d18680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.479+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55cff4e93200 tx=0x55cff4e664e0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 shutdown_connections 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4f22000 0x55cff4d19700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/3008694803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4e81400 0x55cff4d18680 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 >> 172.21.15.191:0/3008694803 conn(0x55cff4e81000 msgr2=0x55cff4b1c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 shutdown_connections 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/3008694803 wait complete. 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8000 msgr2=0x55cff4d17080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8000 0x55cff4d17080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55cff58baf90 tx=0x55cff4d7cb40 comp rx=0 tx=0).stop 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 msgr2=0x55cff4d16580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55cff4d11f80 tx=0x55cff4b1f920 comp rx=0 tx=0).stop 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 shutdown_connections 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cff4db8000 0x55cff4d17080 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 --2- 172.21.15.191:0/1772599675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cff4a2e800 0x55cff4d16580 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 >> 172.21.15.191:0/1772599675 conn(0x55cff4a2e400 msgr2=0x55cff4b1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 shutdown_connections 2024-04-27T04:35:06.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.480+0000 7fabd7d89a80 1 -- 172.21.15.191:0/1772599675 wait complete. 2024-04-27T04:35:06.496 INFO:teuthology.orchestra.run.smithi191.stdout:fooFoofooFooFULL_CONTROL 2024-04-27T04:35:06.497 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-04-27T04:35:06.499 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-04-27T04:35:06.505 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:06.505 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T04:35:06.505 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T04:35:06.506 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-27T04:35:06.552 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:06.553 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.551+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.551+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bfe39a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x561dbb316c00 0x561dbb5fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bfe39a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561dbb641500 con 0x561dbb316800 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bfe39a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x561dbb3b8a00 con 0x561dbb316c00 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bdde3700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x561dbb316800 0x561dbb402800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57116/0 (socket says 172.21.15.191:57116) 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bdde3700 1 -- 172.21.15.191:0/2695887050 learned_addr learned my addr 172.21.15.191:0/2695887050 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:06.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/2695887050 >> v2:172.21.15.138:3300/0 conn(0x561dbb316c00 0x561dbb5fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bcde1700 1 -- 172.21.15.191:0/2695887050 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963305150 0 0) 0x561dbc0e4000 con 0x561dbb316800 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bcde1700 1 -- 172.21.15.191:0/2695887050 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x561dbb641680 con 0x561dbb316800 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bd5e2700 1 -- 172.21.15.191:0/2695887050 >> v1:172.21.15.138:6789/0 conn(0x561dbb316800 legacy=0x561dbb402800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bd5e2700 1 -- 172.21.15.191:0/2695887050 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33fd40 con 0x561dbb316c00 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/2695887050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316c00 0x561dbb5fe580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x561dbb5f9890 tx=0x561dbb407980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1171972a2dd856 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bcde1700 1 -- 172.21.15.191:0/2695887050 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf4a0 con 0x561dbb316c00 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bcde1700 1 -- 172.21.15.191:0/2695887050 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb641500 con 0x561dbb316c00 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.552+0000 7f59bcde1700 1 -- 172.21.15.191:0/2695887050 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf600 con 0x561dbb316c00 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2695887050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316c00 msgr2=0x561dbb5fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/2695887050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316c00 0x561dbb5fe580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x561dbb5f9890 tx=0x561dbb407980 comp rx=0 tx=0).stop 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2695887050 shutdown_connections 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/2695887050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316c00 0x561dbb5fe580 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.578 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2695887050 >> 172.21.15.191:0/2695887050 conn(0x561dbb316400 msgr2=0x561dbb402000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2695887050 shutdown_connections 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2695887050 wait complete. 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.553+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bfe39a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bfe39a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbb3b8500 con 0x561dbb316800 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bdde3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bdde3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54426/0 (socket says 172.21.15.191:54426) 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bdde3700 1 -- 172.21.15.191:0/941012617 learned_addr learned my addr 172.21.15.191:0/941012617 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bdde3700 1 -- 172.21.15.191:0/941012617 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33fba0 con 0x561dbb316800 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bdde3700 1 --2- 172.21.15.191:0/941012617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x561dbc0e7d70 tx=0x561dbb407b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656e4310be32848e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bc5e0700 1 -- 172.21.15.191:0/941012617 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf4a0 con 0x561dbb316800 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bc5e0700 1 -- 172.21.15.191:0/941012617 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb641b00 con 0x561dbb316800 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.554+0000 7f59bc5e0700 1 -- 172.21.15.191:0/941012617 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf600 con 0x561dbb316800 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- 172.21.15.191:0/941012617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 msgr2=0x561dbb5fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/941012617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x561dbc0e7d70 tx=0x561dbb407b60 comp rx=0 tx=0).stop 2024-04-27T04:35:06.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- 172.21.15.191:0/941012617 shutdown_connections 2024-04-27T04:35:06.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/941012617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- 172.21.15.191:0/941012617 >> 172.21.15.191:0/941012617 conn(0x561dbb316400 msgr2=0x561dbb403000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- 172.21.15.191:0/941012617 shutdown_connections 2024-04-27T04:35:06.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- 172.21.15.191:0/941012617 wait complete. 2024-04-27T04:35:06.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bfe39a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbb3b83c0 con 0x561dbb316800 2024-04-27T04:35:06.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bdde3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bdde3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54434/0 (socket says 172.21.15.191:54434) 2024-04-27T04:35:06.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.555+0000 7f59bdde3700 1 -- 172.21.15.191:0/1793164350 learned_addr learned my addr 172.21.15.191:0/1793164350 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59bdde3700 1 -- 172.21.15.191:0/1793164350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33fa00 con 0x561dbb316800 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59bdde3700 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x561dbc0e7e60 tx=0x561dbb407920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59baddd700 1 -- 172.21.15.191:0/1793164350 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf8c0 con 0x561dbb316800 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59baddd700 1 -- 172.21.15.191:0/1793164350 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb3b7c80 con 0x561dbb316800 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59baddd700 1 -- 172.21.15.191:0/1793164350 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf4a0 con 0x561dbb316800 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561dbb33f860 con 0x561dbb316800 2024-04-27T04:35:06.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dbb33f6c0 con 0x561dbb316800 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.556+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59bfe39a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59bfe39a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbb352640 con 0x561dbb317400 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59baddd700 1 -- 172.21.15.191:0/1793164350 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561dbc0b8b00 con 0x561dbb316800 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59baddd700 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6000 0x561dbb5ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59baddd700 1 -- 172.21.15.191:0/1793164350 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x561dbb391180 con 0x561dbb316800 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59be5e4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59be5e4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54436/0 (socket says 172.21.15.191:54436) 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59be5e4700 1 -- 172.21.15.191:0/2354323900 learned_addr learned my addr 172.21.15.191:0/2354323900 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59bdde3700 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6000 0x561dbb5ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59be5e4700 1 -- 172.21.15.191:0/2354323900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33e000 con 0x561dbb317400 2024-04-27T04:35:06.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59bdde3700 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6000 0x561dbb5ff080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x561dbc1a5500 tx=0x561dbb662240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59be5e4700 1 --2- 172.21.15.191:0/2354323900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x561dbc0e7e90 tx=0x561dbb662de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fb6a7ee22cd272 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59b85d8700 1 -- 172.21.15.191:0/2354323900 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf8c0 con 0x561dbb317400 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59b85d8700 1 -- 172.21.15.191:0/2354323900 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb640480 con 0x561dbb317400 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.557+0000 7f59b85d8700 1 -- 172.21.15.191:0/2354323900 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf4a0 con 0x561dbb317400 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2354323900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 msgr2=0x561dbb5feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/2354323900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x561dbc0e7e90 tx=0x561dbb662de0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2354323900 shutdown_connections 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/2354323900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2354323900 >> 172.21.15.191:0/2354323900 conn(0x561dbb316c00 msgr2=0x561dbb403800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2354323900 shutdown_connections 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- 172.21.15.191:0/2354323900 wait complete. 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59bfe39a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbb3528c0 con 0x561dbb317400 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59be5e4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59be5e4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54446/0 (socket says 172.21.15.191:54446) 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.558+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 learned_addr learned my addr 172.21.15.191:0/1598052580 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33eea0 con 0x561dbb317400 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59be5e4700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x561dbc1a43f0 tx=0x561dbb407f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59b6dd5700 1 -- 172.21.15.191:0/1598052580 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cfb80 con 0x561dbb317400 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59b6dd5700 1 -- 172.21.15.191:0/1598052580 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb640180 con 0x561dbb317400 2024-04-27T04:35:06.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59b6dd5700 1 -- 172.21.15.191:0/1598052580 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf8c0 con 0x561dbb317400 2024-04-27T04:35:06.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561dbb33e820 con 0x561dbb317400 2024-04-27T04:35:06.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.559+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dbb33e9c0 con 0x561dbb317400 2024-04-27T04:35:06.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59b6dd5700 1 -- 172.21.15.191:0/1598052580 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561dbc0b8dc0 con 0x561dbb317400 2024-04-27T04:35:06.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59b6dd5700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6c00 0x561dbb5ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59b6dd5700 1 -- 172.21.15.191:0/1598052580 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x561dbb391340 con 0x561dbb317400 2024-04-27T04:35:06.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561dbb317800 0x561dbb5ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6c00 0x561dbb5ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb317c00 con 0x561dbb317800 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59be5e4700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561dbb317800 0x561dbb5ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59be5e4700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561dbb317800 0x561dbb5ffb80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.560+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6c00 0x561dbb5ff600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x561dbb5f9890 tx=0x561dbb407800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 realm 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7000 con 0x561dbb317800 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:06.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7400 con 0x561dbb317800 2024-04-27T04:35:06.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7800 con 0x561dbb317800 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561dbb6a7c00 0x561dbb600100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.561+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7000 con 0x561dbb6a7c00 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bdde3700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561dbb6a7c00 0x561dbb600100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bdde3700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561dbb6a7c00 0x561dbb600100 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bfe39a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bfe39a80 20 searching for the correct realm 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7400 con 0x561dbb6a7c00 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.562+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bdde3700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561dbb6a7400 0x561dbb600c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7000 con 0x561dbb6a7400 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561dbb6a7400 0x561dbb600c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561dbb6a7400 0x561dbb600c00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561dbb6a7000 0x561dbb601180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.563+0000 7f59be5e4700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561dbb6a7000 0x561dbb601180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561dbb6a7000 0x561dbb601180 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.564+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7400 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.565+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7400 2024-04-27T04:35:06.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.566+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.567+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7400 2024-04-27T04:35:06.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.568+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7400 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7400 2024-04-27T04:35:06.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.569+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7c00 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7400 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bfe39a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bfe39a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:06.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bfe39a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bfe39a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.570+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb317c00 con 0x561dbb317800 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a400 con 0x561dbb317800 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a800 con 0x561dbb6a7c00 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:06.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 zone default found 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 4 Realm: () 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 zonegroup default 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76ac00 con 0x561dbb6a7000 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.571+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b000 con 0x561dbb6a7c00 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b400 con 0x561dbb6a7c00 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 20 started sync module instance, tier type = 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.572+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b800 con 0x561dbb6a7400 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.573+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.573+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94685697628928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a800 con 0x561dbb6a7400 2024-04-27T04:35:06.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.573+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b000 con 0x561dbb317800 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.574+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94685697628928] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.574+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.574+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94685697630080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b400 con 0x561dbb317800 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.574+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a400 con 0x561dbb6a7c00 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.575+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.575+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94685697631232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76ac00 con 0x561dbb6a7c00 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.575+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb317c00 con 0x561dbb6a7400 2024-04-27T04:35:06.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.575+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94685697630080] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.576+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.576+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94685697632384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb6a7800 con 0x561dbb6a7400 2024-04-27T04:35:06.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.576+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb317800 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.576+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94685697631232] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.577+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94685697632384] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.577+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.577+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94685697633536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b800 con 0x561dbb317800 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.577+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a800 con 0x561dbb6a7000 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.578+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94685697633536] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.578+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.578+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94685697634688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76bc00 con 0x561dbb6a7000 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.578+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b000 con 0x561dbb6a7000 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.585+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(56 notify.6 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.585+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94685697634688] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.585+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94685697635840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b400 con 0x561dbb6a7000 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.585+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a000 con 0x561dbb6a7000 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94685697635840] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94685697636992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76b800 con 0x561dbb6a7000 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=0 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=1 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=2 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=3 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=4 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=5 2024-04-27T04:35:06.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.586+0000 7f59bfe39a80 20 add_watcher() i=6 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.587+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94685697636992] v33'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.587+0000 7f59bfe39a80 20 add_watcher() i=7 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.587+0000 7f59bfe39a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.587+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76a800 con 0x561dbb317800 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.588+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.588+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94685699203072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb76bc00 con 0x561dbb317800 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94685699203072] v33'186 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59a4db1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bfe39a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bfe39a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bfe39a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bfe39a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbb3b83c0 con 0x561dbb76b400 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bd5e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bd5e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54456/0 (socket says 172.21.15.191:54456) 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.589+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1803668927 learned_addr learned my addr 172.21.15.191:0/1803668927 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.590+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1803668927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33eb60 con 0x561dbb76b400 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.590+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/1803668927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x561dbb70d6b0 tx=0x561dbb4079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8badc6d9ccff2230 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.590+0000 7f59a45b0700 1 -- 172.21.15.191:0/1803668927 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb3cf760 con 0x561dbb76b400 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.590+0000 7f59a45b0700 1 -- 172.21.15.191:0/1803668927 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb78e000 con 0x561dbb76b400 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.590+0000 7f59a45b0700 1 -- 172.21.15.191:0/1803668927 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb790000 con 0x561dbb76b400 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1803668927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 msgr2=0x561dbb600680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1803668927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x561dbb70d6b0 tx=0x561dbb4079e0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1803668927 shutdown_connections 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1803668927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1803668927 >> 172.21.15.191:0/1803668927 conn(0x561dbb76b000 msgr2=0x561dbb404000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1803668927 shutdown_connections 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1803668927 wait complete. 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 Processor -- start 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- start start 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bfe39a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbb3b8500 con 0x561dbb76b400 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bd5e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bd5e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54468/0 (socket says 172.21.15.191:54468) 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.591+0000 7f59bd5e2700 1 -- 172.21.15.191:0/874769574 learned_addr learned my addr 172.21.15.191:0/874769574 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.623 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59bd5e2700 1 -- 172.21.15.191:0/874769574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbb33ed00 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59bd5e2700 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x561dbb77f260 tx=0x561dbb7508a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59a2dad700 1 -- 172.21.15.191:0/874769574 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb790160 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59a2dad700 1 -- 172.21.15.191:0/874769574 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561dbb78e180 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561dbb33e340 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59a2dad700 1 -- 172.21.15.191:0/874769574 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561dbb7902c0 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.592+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dbb33fd40 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.593+0000 7f59a2dad700 1 -- 172.21.15.191:0/874769574 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561dbc0b9b80 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.593+0000 7f59a2dad700 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb808000 0x561dbb601700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.593+0000 7f59a2dad700 1 -- 172.21.15.191:0/874769574 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x561dbb391500 con 0x561dbb76b400 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.593+0000 7f59bfe39a80 5 note: GC not initialized 2024-04-27T04:35:06.624 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.593+0000 7f59be5e4700 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb808000 0x561dbb601700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.593+0000 7f59be5e4700 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb808000 0x561dbb601700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x561dbb68d590 tx=0x561dbb7505a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59a05a8700 20 reqs_thread_entry: start 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f599e5a4700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:37 2024 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f599dd82700 20 reqs_thread_entry: start 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb808400 con 0x561dbb6a7400 2024-04-27T04:35:06.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 : miss 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.594+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.d 8:b151d69a:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb808800 con 0x561dbb6a7400 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 info.flags=0x17 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 to cache LRU end 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:06.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:06.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 2024-04-27T04:35:06.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 20 get_obj_state: rctx=0x561dbb721dc0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x561dbb8781e8 s->prefetch_data=0 2024-04-27T04:35:06.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 3.17s0 3:e9a98dfd:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb808c00 con 0x561dbb317800 2024-04-27T04:35:06.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+926 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 10 manifest: total_size = 5 2024-04-27T04:35:06.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.595+0000 7f59bfe39a80 20 get_obj_state: setting s->obj_tag to _U-1lTSstMZ6jEBmO1f_hJ0-f3TMSHNO 2024-04-27T04:35:06.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.596+0000 7f59a4db1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.596+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94685699203072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb809800 con 0x561dbb317800 2024-04-27T04:35:06.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.597+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94685699203072] v33'187 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.597+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94685697628928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb809c00 con 0x561dbb6a7400 2024-04-27T04:35:06.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.598+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94685697628928] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.598+0000 7f59bfe39a80 20 remove_watcher() i=0 2024-04-27T04:35:06.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.598+0000 7f59bfe39a80 2 removed watcher, disabling cache 2024-04-27T04:35:06.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.598+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94685697630080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb808400 con 0x561dbb317800 2024-04-27T04:35:06.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.599+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94685697630080] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.599+0000 7f59bfe39a80 20 remove_watcher() i=1 2024-04-27T04:35:06.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.599+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94685697631232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb808800 con 0x561dbb6a7c00 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.600+0000 7f59bdde3700 1 -- 172.21.15.191:0/1598052580 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94685697631232] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f200 con 0x561dbb6a7c00 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.600+0000 7f59bfe39a80 20 remove_watcher() i=2 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.600+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94685697632384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb809c00 con 0x561dbb6a7400 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.601+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94685697632384] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75f440 con 0x561dbb6a7400 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.601+0000 7f59bfe39a80 20 remove_watcher() i=3 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.601+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94685697633536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb809400 con 0x561dbb317800 2024-04-27T04:35:06.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.602+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94685697633536] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb317800 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.602+0000 7f59bfe39a80 20 remove_watcher() i=4 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.602+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94685697634688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb809000 con 0x561dbb6a7000 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.603+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94685697634688] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.603+0000 7f59bfe39a80 20 remove_watcher() i=5 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.603+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94685697635840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb317c00 con 0x561dbb6a7000 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.604+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94685697635840] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.604+0000 7f59bfe39a80 20 remove_watcher() i=6 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.604+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94685697636992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561dbb317c00 con 0x561dbb6a7000 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59be5e4700 1 -- 172.21.15.191:0/1598052580 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94685697636992] v33'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561dbb75eb40 con 0x561dbb6a7000 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 20 remove_watcher() i=7 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561dbb6a7000 msgr2=0x561dbb601180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x561dbb6a7000 0x561dbb601180 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561dbb317800 msgr2=0x561dbb5ffb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x561dbb317800 0x561dbb5ffb80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561dbb6a7400 msgr2=0x561dbb600c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x561dbb6a7400 0x561dbb600c00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561dbb6a7c00 msgr2=0x561dbb600100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.605+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x561dbb6a7c00 0x561dbb600100 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6c00 msgr2=0x561dbb5ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6c00 0x561dbb5ff600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x561dbb5f9890 tx=0x561dbb407800 comp rx=0 tx=0).stop 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 msgr2=0x561dbb5feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1598052580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb317400 0x561dbb5feb00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x561dbc1a43f0 tx=0x561dbb407f20 comp rx=0 tx=0).stop 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 reap_dead start 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bd5e2700 1 -- 172.21.15.191:0/1598052580 reap_dead start 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 shutdown_connections 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 >> 172.21.15.191:0/1598052580 conn(0x561dbb316c00 msgr2=0x561dbb403800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 shutdown_connections 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1598052580 wait complete. 2024-04-27T04:35:06.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb808000 msgr2=0x561dbb601700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb808000 0x561dbb601700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x561dbb68d590 tx=0x561dbb7505a0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 msgr2=0x561dbb600680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x561dbb77f260 tx=0x561dbb7508a0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 shutdown_connections 2024-04-27T04:35:06.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb808000 0x561dbb601700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/874769574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb76b400 0x561dbb600680 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 >> 172.21.15.191:0/874769574 conn(0x561dbb76b000 msgr2=0x561dbb404000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 shutdown_connections 2024-04-27T04:35:06.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/874769574 wait complete. 2024-04-27T04:35:06.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6000 msgr2=0x561dbb5ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6000 0x561dbb5ff080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x561dbc1a5500 tx=0x561dbb662240 comp rx=0 tx=0).stop 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 msgr2=0x561dbb5fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.606+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x561dbc0e7e60 tx=0x561dbb407920 comp rx=0 tx=0).stop 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.607+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 shutdown_connections 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.607+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x561dbb6a6000 0x561dbb5ff080 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.607+0000 7f59bfe39a80 1 --2- 172.21.15.191:0/1793164350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dbb316800 0x561dbb5fe580 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.607+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 >> 172.21.15.191:0/1793164350 conn(0x561dbb316400 msgr2=0x561dbb403000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.607+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 shutdown_connections 2024-04-27T04:35:06.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.607+0000 7f59bfe39a80 1 -- 172.21.15.191:0/1793164350 wait complete. 2024-04-27T04:35:06.638 INFO:teuthology.orchestra.run.smithi191.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-27T04:35:06.638 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-27T04:35:06.641 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-04-27T04:35:06.644 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:06.739 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-27T04:35:06.739 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-27T04:35:06.739 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T04:35:06.739 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-27T04:35:06.739 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-27T04:35:06.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-27T04:35:06.740 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-04-27T04:35:06.788 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:06.788 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:06.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.787+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.787+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.787+0000 7fa3d0117a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x562f48212c00 0x562f484fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.787+0000 7fa3d0117a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562f4853d500 con 0x562f48212800 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.787+0000 7fa3d0117a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x562f482b4a00 con 0x562f48212c00 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd8c0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x562f48212c00 0x562f484fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd8c0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x562f48212c00 0x562f484fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54482/0 (socket says 172.21.15.191:54482) 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3ce0c1700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x562f48212800 0x562f482fe800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57120/0 (socket says 172.21.15.191:57120) 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/3905553027 learned_addr learned my addr 172.21.15.191:0/3905553027 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/3905553027 >> v1:172.21.15.138:6789/0 conn(0x562f48212800 legacy=0x562f482fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/3905553027 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823bd40 con 0x562f48212c00 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/3905553027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212c00 0x562f484fa580 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x562f484f5890 tx=0x562f48303920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c731cc1c10805a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd0bf700 1 -- 172.21.15.191:0/3905553027 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb4a0 con 0x562f48212c00 2024-04-27T04:35:06.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd0bf700 1 -- 172.21.15.191:0/3905553027 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f4853d800 con 0x562f48212c00 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.788+0000 7fa3cd0bf700 1 -- 172.21.15.191:0/3905553027 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb600 con 0x562f48212c00 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3905553027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212c00 msgr2=0x562f484fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/3905553027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212c00 0x562f484fa580 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x562f484f5890 tx=0x562f48303920 comp rx=0 tx=0).stop 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3905553027 shutdown_connections 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/3905553027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212c00 0x562f484fa580 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3905553027 >> 172.21.15.191:0/3905553027 conn(0x562f48212400 msgr2=0x562f482fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3905553027 shutdown_connections 2024-04-27T04:35:06.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3905553027 wait complete. 2024-04-27T04:35:06.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.789+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3d0117a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3d0117a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f482b4500 con 0x562f48212800 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54494/0 (socket says 172.21.15.191:54494) 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/2633753729 learned_addr learned my addr 172.21.15.191:0/2633753729 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/2633753729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823bba0 con 0x562f48212800 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/2633753729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x562f48fd9890 tx=0x562f48303b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29032b507a18b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3cc8be700 1 -- 172.21.15.191:0/2633753729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb1e0 con 0x562f48212800 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3cc8be700 1 -- 172.21.15.191:0/2633753729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f4853db00 con 0x562f48212800 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.790+0000 7fa3cc8be700 1 -- 172.21.15.191:0/2633753729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb4a0 con 0x562f48212800 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2633753729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 msgr2=0x562f484fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/2633753729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x562f48fd9890 tx=0x562f48303b60 comp rx=0 tx=0).stop 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2633753729 shutdown_connections 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/2633753729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2633753729 >> 172.21.15.191:0/2633753729 conn(0x562f48212400 msgr2=0x562f482ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2633753729 shutdown_connections 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2633753729 wait complete. 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3d0117a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f482b43c0 con 0x562f48212800 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54504/0 (socket says 172.21.15.191:54504) 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.791+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/3113385927 learned_addr learned my addr 172.21.15.191:0/3113385927 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/3113385927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823ba00 con 0x562f48212800 2024-04-27T04:35:06.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x562f48fd96b0 tx=0x562f48303a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3cb0bb700 1 -- 172.21.15.191:0/3113385927 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb600 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3cb0bb700 1 -- 172.21.15.191:0/3113385927 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f482b3c80 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3cb0bb700 1 -- 172.21.15.191:0/3113385927 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb1e0 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f4823b860 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f4823b6c0 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.792+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3cb0bb700 1 -- 172.21.15.191:0/3113385927 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562f48fb4b00 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3cb0bb700 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48213400 0x562f484fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3d0117a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3cb0bb700 1 -- 172.21.15.191:0/3113385927 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x562f4828d180 con 0x562f48212800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3d0117a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4824e640 con 0x562f48213800 2024-04-27T04:35:06.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48213400 0x562f484fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54506/0 (socket says 172.21.15.191:54506) 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/2526855229 learned_addr learned my addr 172.21.15.191:0/2526855229 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/2526855229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823a000 con 0x562f48213800 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48213400 0x562f484fb080 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x562f49090ae0 tx=0x562f4855ede0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/2526855229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x562f490910e0 tx=0x562f4855e240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdce86fe8cca5299 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3c88b6700 1 -- 172.21.15.191:0/2526855229 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb4a0 con 0x562f48213800 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3c88b6700 1 -- 172.21.15.191:0/2526855229 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f4853c300 con 0x562f48213800 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.793+0000 7fa3c88b6700 1 -- 172.21.15.191:0/2526855229 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb8c0 con 0x562f48213800 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2526855229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 msgr2=0x562f484fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/2526855229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x562f490910e0 tx=0x562f4855e240 comp rx=0 tx=0).stop 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2526855229 shutdown_connections 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/2526855229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2526855229 >> 172.21.15.191:0/2526855229 conn(0x562f48212c00 msgr2=0x562f482ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2526855229 shutdown_connections 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- 172.21.15.191:0/2526855229 wait complete. 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3d0117a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4824e8c0 con 0x562f48213800 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3ce0c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54522/0 (socket says 172.21.15.191:54522) 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.794+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 learned_addr learned my addr 172.21.15.191:0/4064534253 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823aea0 con 0x562f48213800 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x562f490902d0 tx=0x562f4855efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3c70b3700 1 -- 172.21.15.191:0/4064534253 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb4a0 con 0x562f48213800 2024-04-27T04:35:06.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3c70b3700 1 -- 172.21.15.191:0/4064534253 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f482b3e00 con 0x562f48213800 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f4823a820 con 0x562f48213800 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3c70b3700 1 -- 172.21.15.191:0/4064534253 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb8c0 con 0x562f48213800 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.795+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f4823a9c0 con 0x562f48213800 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3c70b3700 1 -- 172.21.15.191:0/4064534253 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562f48fb4dc0 con 0x562f48213800 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3c70b3700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f4908cc00 0x562f484fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3c70b3700 1 -- 172.21.15.191:0/4064534253 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x562f4828d340 con 0x562f48213800 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f4908cc00 0x562f484fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f48213c00 0x562f484fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d000 con 0x562f48213c00 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f48213c00 0x562f484fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f4908cc00 0x562f484fb600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x562f484f5890 tx=0x562f48303800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.796+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f48213c00 0x562f484fbb80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 realm 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d400 con 0x562f48213c00 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d800 con 0x562f48213c00 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908dc00 con 0x562f48213c00 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f4908d000 0x562f484fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.797+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d400 con 0x562f4908d000 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f4908d000 0x562f484fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f4908d000 0x562f484fc100 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3d0117a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3d0117a80 20 searching for the correct realm 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640000 con 0x562f4908d000 2024-04-27T04:35:06.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.798+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48640000 0x562f484fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d400 con 0x562f48640000 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48640000 0x562f484fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48640000 0x562f484fcc00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f4908d400 0x562f484fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.799+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f4908d400 0x562f484fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f4908d400 0x562f484fd180 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.800+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.801+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.802+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.803+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.804+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.805+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.806+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.806+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d400 2024-04-27T04:35:06.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.806+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.806+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.806+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.806+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640400 con 0x562f48213c00 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f48213c00 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640c00 con 0x562f4908d000 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 zone default found 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 4 Realm: () 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:06.829 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 zonegroup default 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.807+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641000 con 0x562f4908d400 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f4908d000 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640400 con 0x562f4908d000 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 started sync module instance, tier type = 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.808+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f48640000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.809+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.809+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94761076595456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641000 con 0x562f48640000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.809+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641c00 con 0x562f48213c00 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.810+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94761076595456] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.811+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.811+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94761076596608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908dc00 con 0x562f48213c00 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.811+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d800 con 0x562f4908d000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.811+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.811+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94761076597760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.812+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f48640000 2024-04-27T04:35:06.831 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.812+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94761076596608] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.812+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94761076597760] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.813+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.813+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94761076598912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641000 con 0x562f48640000 2024-04-27T04:35:06.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.813+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640c00 con 0x562f48213c00 2024-04-27T04:35:06.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.814+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94761076598912] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.814+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.814+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94761076600064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f48213c00 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.814+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640400 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.815+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.815+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94761076601216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.815+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94761076600064] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.815+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.816+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94761076601216] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.816+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.816+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94761076602368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.816+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e800 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94761076602368] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94761076603520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f4908d400 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=0 2024-04-27T04:35:06.843 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=1 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=2 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=3 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=4 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=5 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.817+0000 7fa3d0117a80 20 add_watcher() i=6 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.818+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94761076603520] v33'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.818+0000 7fa3d0117a80 20 add_watcher() i=7 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.818+0000 7fa3d0117a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.818+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f000 con 0x562f48213c00 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.818+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.818+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94761078054912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f400 con 0x562f48213c00 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.819+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94761078054912] v33'188 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.819+0000 7fa3b508f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.844 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.819+0000 7fa3d0117a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.819+0000 7fa3d0117a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3d0117a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3d0117a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f482b43c0 con 0x562f4864fc00 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3cd8c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3cd8c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54524/0 (socket says 172.21.15.191:54524) 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/580473851 learned_addr learned my addr 172.21.15.191:0/580473851 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/580473851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823ab60 con 0x562f4864fc00 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.820+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/580473851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x562f485d56e0 tx=0x562f48303980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df3c52c865d76cc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3b488e700 1 -- 172.21.15.191:0/580473851 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f482cb760 con 0x562f4864fc00 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3b488e700 1 -- 172.21.15.191:0/580473851 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f48664000 con 0x562f4864fc00 2024-04-27T04:35:06.845 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3b488e700 1 -- 172.21.15.191:0/580473851 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f48654160 con 0x562f4864fc00 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 -- 172.21.15.191:0/580473851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 msgr2=0x562f484fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/580473851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x562f485d56e0 tx=0x562f48303980 comp rx=0 tx=0).stop 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 -- 172.21.15.191:0/580473851 shutdown_connections 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/580473851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 -- 172.21.15.191:0/580473851 >> 172.21.15.191:0/580473851 conn(0x562f4864f800 msgr2=0x562f48300000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 -- 172.21.15.191:0/580473851 shutdown_connections 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.821+0000 7fa3d0117a80 1 -- 172.21.15.191:0/580473851 wait complete. 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3d0117a80 1 Processor -- start 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3d0117a80 1 -- start start 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3d0117a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3d0117a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f482b4500 con 0x562f4864fc00 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3cd8c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.846 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3cd8c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54530/0 (socket says 172.21.15.191:54530) 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 learned_addr learned my addr 172.21.15.191:0/1357579474 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4823ad00 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x562f4865a810 tx=0x562f4861c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3b308b700 1 -- 172.21.15.191:0/1357579474 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f486542c0 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3b308b700 1 -- 172.21.15.191:0/1357579474 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f48664180 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3b308b700 1 -- 172.21.15.191:0/1357579474 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f48654420 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f4823a340 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.822+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f4823bd40 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.823+0000 7fa3b308b700 1 -- 172.21.15.191:0/1357579474 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562f48fb5b80 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.823+0000 7fa3b308b700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48641c00 0x562f484fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.823+0000 7fa3b308b700 1 -- 172.21.15.191:0/1357579474 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x562f4828d6c0 con 0x562f4864fc00 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.823+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48641c00 0x562f484fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.823+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f4908dc00 0x562f486f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.823+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f4908dc00 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.824+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f4908dc00 0x562f486f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.824+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48641c00 0x562f484fd700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x562f485d5ef0 tx=0x562f4861d5c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.824+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f4908dc00 0x562f486f4000 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.825+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'57 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908dc00 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.825+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f48640c00 0x562f486f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.825+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f48640c00 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.825+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f48640c00 0x562f486f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.825+0000 7fa3cd8c0700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f48640c00 0x562f486f4580 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.826+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.826+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f48640400 0x562f486f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.826+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644000 con 0x562f48640400 2024-04-27T04:35:06.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.826+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f48640400 0x562f486f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.827+0000 7fa3ce8c2700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f48640400 0x562f486f4b00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.827+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.827+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644400 con 0x562f4908dc00 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.828+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'189 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908dc00 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.828+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48644800 0x562f486f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.828+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644c00 con 0x562f48644800 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.829+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48644800 0x562f486f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.829+0000 7fa3ce0c1700 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48644800 0x562f486f5080 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.830+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'71 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.830+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645000 con 0x562f48640400 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.831+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'54 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.831+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645400 con 0x562f4908dc00 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.832+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908dc00 2024-04-27T04:35:06.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.832+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645800 con 0x562f48640c00 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.833+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'49 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.833+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f48644800 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.834+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.834+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644400 con 0x562f48640400 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.835+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.835+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644c00 con 0x562f48644800 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.835+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'72 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.835+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644000 con 0x562f48640c00 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.836+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'29 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.836+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f48644800 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.837+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.837+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f48640c00 2024-04-27T04:35:06.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.839+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'64 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.839+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e800 con 0x562f4908dc00 2024-04-27T04:35:06.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.840+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'190 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908dc00 2024-04-27T04:35:06.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.840+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f48640400 2024-04-27T04:35:06.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.841+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'62 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.841+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f000 con 0x562f48644800 2024-04-27T04:35:06.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.842+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'73 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.842+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f400 con 0x562f48640400 2024-04-27T04:35:06.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.843+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'36 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.843+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d800 con 0x562f4908dc00 2024-04-27T04:35:06.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.844+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908dc00 2024-04-27T04:35:06.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.844+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f4908dc00 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.845+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'191 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908dc00 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.845+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.846+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'63 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.846+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645400 con 0x562f48644800 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.847+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'40 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.847+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.848+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.848+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644c00 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.849+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'42 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.849+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645800 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.850+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.850+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644000 con 0x562f48640400 2024-04-27T04:35:06.855 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.851+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'55 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.851+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f48640c00 2024-04-27T04:35:06.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.852+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.852+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f48640400 2024-04-27T04:35:06.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.853+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'36 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.856 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.853+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f48640c00 2024-04-27T04:35:06.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.854+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'50 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.854+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e800 con 0x562f48640400 2024-04-27T04:35:06.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.854+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/1357579474 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'64 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48640400 2024-04-27T04:35:06.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.854+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f000 con 0x562f48640c00 2024-04-27T04:35:06.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.855+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/1357579474 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'65 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640c00 2024-04-27T04:35:06.857 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.856+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645000 con 0x562f48644800 2024-04-27T04:35:06.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.856+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.857+0000 7fa3b0886700 20 reqs_thread_entry: start 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.857+0000 7fa3d0117a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3ae882700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:37 2024 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3ae060700 20 reqs_thread_entry: start 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f48640000 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T04:35:06.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 : miss 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.d 8:b151d69a:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644c00 con 0x562f48640000 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(63 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 info.flags=0x17 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 to cache LRU end 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.858+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 4.17 4:e87efb20:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645800 con 0x562f4908d400 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.859+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(64 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.859+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 4.c 4:30d32623:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48213c00 2024-04-27T04:35:06.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.859+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(65 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.859+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 4.17 4:e8418f67:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f4908d400 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.860+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 [call out=425b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.860+0000 7fa3d0117a80 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.860+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 4.5 4:a12347d4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f400 con 0x562f4908d000 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.860+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.860+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 4.f 4:f0befbc0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f48213c00 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 4.2 4:417c88b7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f48213c00 2024-04-27T04:35:06.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 4.24 4:250e3888:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f4908d400 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 [call out=425b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3d0117a80 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.861+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 4.13 4:c9de2384:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f000 con 0x562f48640000 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.862+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.862+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 4.24 4:251ae78f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f4908d400 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.862+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 [call out=425b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.862+0000 7fa3d0117a80 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.862+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 4.39 4:9e7ecd5a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e800 con 0x562f4908d000 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f4908d400 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [call out=698b] v0'0 uv11 ondisk = 0) v8 ==== 195+0+698 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 3.as0 3:52c0c3c2:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f4908d000 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(75 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 10 manifest: total_size = 6 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.9270155267653875039 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 20 prepare_atomic_modification: state is not atomic. state=0x562f487a41e8 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.863+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 4.17 4:e8418f67:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f4908d400 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.865+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(76 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 [stat,call,call] v33'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.865+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:77 3.as0 3:52c0c3c2:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f4908d000 2024-04-27T04:35:06.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.866+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(77 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_9c8454ddf7aa50116496bac348d7550d [call] v33'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.866+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 4.17 4:e8418f67:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645800 con 0x562f4908d400 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.866+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.866+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:79 3.39s0 3:9ec2ff2e:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f4908d000 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(79 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3d0117a80 10 manifest: total_size = 3 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3d0117a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.5650942049029810098 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3d0117a80 20 prepare_atomic_modification: state is not atomic. state=0x562f487a41e8 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3d0117a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 2024-04-27T04:35:06.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.867+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:80 4.24 4:250e3888:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f4908d400 2024-04-27T04:35:06.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.869+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(78 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 [stat,call,call] v33'8 uv8 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.869+0000 7fa3c88b6700 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-04-27T04:35:06.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.869+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(80 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 [stat,call,call] v33'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.869+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:81 3.39s0 3:9ec2ff2e:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f4908d000 2024-04-27T04:35:06.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.870+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(81 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_b1b9a972ccd8c962a473909b97007eb4 [call] v33'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.870+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:82 4.24 4:250e3888:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e800 con 0x562f4908d400 2024-04-27T04:35:06.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.870+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.870+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:83 3.34s0 3:2f0fc79b:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f48213c00 2024-04-27T04:35:06.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(83 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=877b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3d0117a80 10 manifest: total_size = 4 2024-04-27T04:35:06.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3d0117a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.2680290792917523796 2024-04-27T04:35:06.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3d0117a80 20 prepare_atomic_modification: state is not atomic. state=0x562f487a41e8 2024-04-27T04:35:06.873 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3d0117a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 2024-04-27T04:35:06.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 4.24 4:251ae78f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645400 con 0x562f4908d400 2024-04-27T04:35:06.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(82 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 [stat,call,call] v33'8 uv8 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.871+0000 7fa3c80b5700 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-04-27T04:35:06.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.872+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(84 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 [stat,call,call] v33'9 uv9 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.874 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.872+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:85 3.34s0 3:2f0fc79b:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645000 con 0x562f48213c00 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(85 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v33'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 4.24 4:251ae78f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f4908d400 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:87 3.3cs0 3:3def1da6:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8000 con 0x562f48640000 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(87 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 10 manifest: total_size = 5 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.17216026774765837394 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 20 prepare_atomic_modification: state is not atomic. state=0x562f487a41e8 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 2024-04-27T04:35:06.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.874+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:88 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8400 con 0x562f4908d400 2024-04-27T04:35:06.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.875+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(86 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 [stat,call,call] v33'10 uv10 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.875+0000 7fa3c88b6700 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-04-27T04:35:06.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.876+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(88 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [stat,call,call] v33'12 uv12 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.876+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:89 3.3cs0 3:3def1da6:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8800 con 0x562f48640000 2024-04-27T04:35:06.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(89 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_24d27c169c2c881eb09a065116f2aa5c [call] v33'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:90 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9000 con 0x562f4908d400 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 3.17s0 3:e9a98dfd:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9400 con 0x562f48213c00 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(91 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+926 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 10 manifest: total_size = 5 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 20 get_obj_state: setting s->obj_tag to _U-1lTSstMZ6jEBmO1f_hJ0-f3TMSHNO 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 20 get_obj_state: rctx=0x562f485ed7a0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x562f487a41e8 s->prefetch_data=0 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 20 prepare_atomic_modification: state is not atomic. state=0x562f487a41e8 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 2024-04-27T04:35:06.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.878+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:92 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9800 con 0x562f4908d400 2024-04-27T04:35:06.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.879+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(90 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [stat,call,call] v33'13 uv13 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.879+0000 7fa3c80b5700 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-04-27T04:35:06.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.879+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== osd_op_reply(92 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [stat,call,call] v33'14 uv14 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.881 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.880+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:93 3.17s0 3:e9a98dfd:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9c00 con 0x562f48213c00 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(93 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5_bb3aec0fdcdbc2974890f805c585d432 [call] v33'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:94 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8400 con 0x562f4908d400 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 20 list_objects_ordered: starting attempt 1 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:95 4.17 4:e87efb20:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f4908d400 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 4.c 4:30d32623:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645800 con 0x562f48213c00 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:97 4.17 4:e8418f67:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645400 con 0x562f4908d400 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:98 4.5 4:a12347d4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645000 con 0x562f4908d000 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:99 4.f 4:f0befbc0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f400 con 0x562f48213c00 2024-04-27T04:35:06.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:100 4.2 4:417c88b7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f48213c00 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 4.24 4:250e3888:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e800 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:102 4.13 4:c9de2384:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f48640000 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:103 4.24 4:251ae78f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:104 4.39 4:9e7ecd5a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f4908d000 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:105 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f400 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== osd_op_reply(95 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== osd_op_reply(101 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 [call out=124b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(100 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(97 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 [call out=124b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.882+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(98 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 33 ==== osd_op_reply(103 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(102 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(96 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(99 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(104 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 34 ==== osd_op_reply(94 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [stat,call,call] v33'15 uv15 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3c80b5700 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 35 ==== osd_op_reply(105 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [call out=124b] v0'0 uv15 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5])"; uploads.size()=0; is_truncated=0 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5]) 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x17, cached=0x11) 2024-04-27T04:35:06.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.883+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f48640000 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.884+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(106 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv35 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.884+0000 7fa3d0117a80 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.884+0000 7fa3d0117a80 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.884+0000 7fa3d0117a80 10 lifecycle: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.884+0000 7fa3d0117a80 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:06.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.884+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:107 8.d 8:b151d69a:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645400 con 0x562f48640000 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(107 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [delete,create,call,call,writefull 0~352,setxattr (123)] v33'5 uv5 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3d0117a80 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 info.flags=0x17 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3d0117a80 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 to cache LRU end 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3d0117a80 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3d0117a80 10 lifecycle: distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5, ofs0, ns] 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 94761078058368 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f48640000 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== watch-notify(notify (1) cookie 94761076598912 notify 141733920792 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x562f486adba0 con 0x562f48640000 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3c80b5700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920792 cookie 94761076598912 notifier 4700 bl.length()=725 2024-04-27T04:35:06.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== osd_op_reply(108 notify.3 [notify cookie 94761078058368 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3c80b5700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 info.flags=0x17 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3c80b5700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 to cache LRU end 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3c80b5700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.885+0000 7fa3c80b5700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8c00 con 0x562f48640000 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== watch-notify(notify_complete (2) cookie 94761078058368 notify 141733920792 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562f486add40 con 0x562f48640000 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3d0117a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5]): orig_sources= new_sources= 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3d0117a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5]): potential sources added=[] removed=[] 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3d0117a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5]): orig_dests= new_dests= 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3d0117a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5]): potential dests added=[] removed=[] 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.886+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 6.15 6:abd89f08:lc::lc.23:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9000 con 0x562f48644800 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.887+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(33 lc.23 [call] v33'34 uv34 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.887+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:34 6.15 6:abd89f08:lc::lc.23:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9400 con 0x562f48644800 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.888+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(34 lc.23 [call] v33'35 uv35 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.888+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:35 6.15 6:abd89f08:lc::lc.23:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9800 con 0x562f48644800 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.888+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(35 lc.23 [call] v33'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f48644800 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.888+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:110 4.17 4:e87efb20:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9c00 con 0x562f4908d400 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.888+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:111 4.c 4:30d32623:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8400 con 0x562f48213c00 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.888+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:112 4.17 4:e8418f67:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f4908d400 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:113 4.5 4:a12347d4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f000 con 0x562f4908d000 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:114 4.f 4:f0befbc0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f48213c00 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:115 4.2 4:417c88b7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f48213c00 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:116 4.24 4:250e3888:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9000 con 0x562f4908d400 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:117 4.13 4:c9de2384:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9400 con 0x562f48640000 2024-04-27T04:35:06.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:118 4.24 4:251ae78f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9800 con 0x562f4908d400 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:119 4.39 4:9e7ecd5a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8000 con 0x562f4908d000 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:120 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8800 con 0x562f4908d400 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(111 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(115 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(114 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(117 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(119 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 36 ==== osd_op_reply(110 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(113 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 37 ==== osd_op_reply(116 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 [call out=124b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 38 ==== osd_op_reply(120 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [call out=124b] v0'0 uv15 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 39 ==== osd_op_reply(118 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 40 ==== osd_op_reply(112 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 [call out=124b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.889+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:121 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f48640000 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.890+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(121 foo.buckets [call] v33'38 uv38 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.890+0000 7fa3d0117a80 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.890+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:122 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645000 con 0x562f48640000 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(122 myfoo [call,call,delete] v33'39 uv39 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3d0117a80 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3d0117a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:123 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94761078058368 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644c00 con 0x562f4908d400 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 41 ==== watch-notify(notify (1) cookie 94761076602368 notify 141733920789 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x562f485d9380 con 0x562f4908d400 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3c80b5700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920789 cookie 94761076602368 notifier 4700 bl.length()=149 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 42 ==== osd_op_reply(123 notify.6 [notify cookie 94761078058368 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.891+0000 7fa3c80b5700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:124 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f4908d400 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.892+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 43 ==== osd_op_reply(124 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.892+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 44 ==== watch-notify(notify_complete (2) cookie 94761078058368 notify 141733920789 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562f485d9380 con 0x562f4908d400 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.892+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:125 8.d 8:b151d69a:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641800 con 0x562f48640000 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 27 ==== osd_op_reply(125 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [call,call,delete] v33'6 uv6 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3d0117a80 10 removing default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 from cache 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3d0117a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5, ofs0, ns] 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:126 7.e 7:7759931f:::notify.3:head [notify cookie 94761078058368 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48641400 con 0x562f48640000 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 28 ==== watch-notify(notify (1) cookie 94761076598912 notify 141733920793 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x562f486add40 con 0x562f48640000 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3c88b6700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920793 cookie 94761076598912 notifier 4700 bl.length()=206 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 29 ==== osd_op_reply(126 notify.3 [notify cookie 94761078058368 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.893+0000 7fa3c88b6700 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:127 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908d800 con 0x562f48640000 2024-04-27T04:35:06.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 30 ==== osd_op_reply(127 notify.3 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 31 ==== watch-notify(notify_complete (2) cookie 94761078058368 notify 141733920793 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562f486ac340 con 0x562f48640000 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:128 4.17 4:e87efb20:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48644c00 con 0x562f4908d400 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:129 4.c 4:30d32623:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645800 con 0x562f48213c00 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:130 4.17 4:e8418f67:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645400 con 0x562f4908d400 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:131 4.5 4:a12347d4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48645c00 con 0x562f4908d000 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:132 4.f 4:f0befbc0:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9c00 con 0x562f48213c00 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:133 4.2 4:417c88b7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8400 con 0x562f48213c00 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:134 4.24 4:250e3888:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9000 con 0x562f4908d400 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:135 4.13 4:c9de2384:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9800 con 0x562f48640000 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:136 4.24 4:251ae78f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8800 con 0x562f4908d400 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:137 4.39 4:9e7ecd5a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a8c00 con 0x562f4908d000 2024-04-27T04:35:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.894+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:138 4.2d 4:b7cb455a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f487a9400 con 0x562f4908d400 2024-04-27T04:35:06.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 45 ==== osd_op_reply(128 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.0 [delete] v33'9 uv9 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 46 ==== osd_op_reply(134 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.6 [delete] v33'11 uv11 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 47 ==== osd_op_reply(138 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.10 [delete] v33'16 uv16 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 32 ==== osd_op_reply(135 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.7 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 34 ==== osd_op_reply(129 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.1 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 35 ==== osd_op_reply(132 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.4 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.895+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 36 ==== osd_op_reply(133 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.5 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.896+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(137 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.9 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.896+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 48 ==== osd_op_reply(130 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.2 [delete] v33'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.896+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(131 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.3 [delete] v33'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.897+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 49 ==== osd_op_reply(136 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5.8 [delete] v33'12 uv12 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.897+0000 7fa3d0117a80 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 : miss 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.897+0000 7fa3d0117a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.897+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:139 6.d 6:b7536d84:::pubsub..bucket.myfoo%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4908c400 con 0x562f48213c00 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.897+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 37 ==== osd_op_reply(139 pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.898+0000 7fa3d0117a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.898+0000 7fa3d0117a80 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 info.flags=0x0 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.898+0000 7fa3d0117a80 10 adding default.rgw.log++pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 to cache LRU end 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.898+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:140 6.d 6:b7536d84:::pubsub..bucket.myfoo%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864f400 con 0x562f48213c00 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.899+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 38 ==== osd_op_reply(140 pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.5 [delete] v33'33 uv29 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.899+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:141 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e000 con 0x562f48640000 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.900+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 33 ==== osd_op_reply(141 foo.buckets [call] v33'40 uv40 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.901+0000 7fa3b508f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.901+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94761078054912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864ec00 con 0x562f48213c00 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.902+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 39 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94761078054912] v33'192 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.902+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:143 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94761076595456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f4864e400 con 0x562f48640000 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.903+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 34 ==== osd_op_reply(143 notify.0 [watch unwatch cookie 94761076595456] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.903+0000 7fa3d0117a80 20 remove_watcher() i=0 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.903+0000 7fa3d0117a80 2 removed watcher, disabling cache 2024-04-27T04:35:06.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.903+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:144 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94761076596608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48640800 con 0x562f48213c00 2024-04-27T04:35:06.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.904+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 40 ==== osd_op_reply(144 notify.1 [watch unwatch cookie 94761076596608] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.904+0000 7fa3d0117a80 20 remove_watcher() i=1 2024-04-27T04:35:06.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.904+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:145 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94761076597760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48624000 con 0x562f4908d000 2024-04-27T04:35:06.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.905+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/4064534253 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(145 notify.2 [watch unwatch cookie 94761076597760] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635200 con 0x562f4908d000 2024-04-27T04:35:06.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.905+0000 7fa3d0117a80 20 remove_watcher() i=2 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.905+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:146 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94761076598912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48624400 con 0x562f48640000 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.906+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 <== osd.2 v2:172.21.15.191:6816/41888 35 ==== osd_op_reply(146 notify.3 [watch unwatch cookie 94761076598912] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48635440 con 0x562f48640000 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.906+0000 7fa3d0117a80 20 remove_watcher() i=3 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.906+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:147 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94761076600064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48624800 con 0x562f48213c00 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.907+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.1 v2:172.21.15.191:6808/41886 41 ==== osd_op_reply(147 notify.4 [watch unwatch cookie 94761076600064] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f48213c00 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.907+0000 7fa3d0117a80 20 remove_watcher() i=4 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.907+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:148 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94761076601216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48624c00 con 0x562f4908d400 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.908+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 50 ==== osd_op_reply(148 notify.5 [watch unwatch cookie 94761076601216] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.908+0000 7fa3d0117a80 20 remove_watcher() i=5 2024-04-27T04:35:06.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.908+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:149 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94761076602368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48625000 con 0x562f4908d400 2024-04-27T04:35:06.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.909+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 51 ==== osd_op_reply(149 notify.6 [watch unwatch cookie 94761076602368] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.909+0000 7fa3d0117a80 20 remove_watcher() i=6 2024-04-27T04:35:06.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.909+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:150 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94761076603520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562f48625400 con 0x562f4908d400 2024-04-27T04:35:06.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3ce8c2700 1 -- 172.21.15.191:0/4064534253 <== osd.0 v2:172.21.15.138:6800/39496 52 ==== osd_op_reply(150 notify.7 [watch unwatch cookie 94761076603520] v33'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f48634b40 con 0x562f4908d400 2024-04-27T04:35:06.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 20 remove_watcher() i=7 2024-04-27T04:35:06.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f4908d400 msgr2=0x562f484fd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f4908d400 0x562f484fd180 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f48213c00 msgr2=0x562f484fbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f48213c00 0x562f484fbb80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48640000 msgr2=0x562f484fcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48640000 0x562f484fcc00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f4908d000 msgr2=0x562f484fc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.910+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f4908d000 0x562f484fc100 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f4908cc00 msgr2=0x562f484fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f4908cc00 0x562f484fb600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x562f484f5890 tx=0x562f48303800 comp rx=0 tx=0).stop 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 msgr2=0x562f484fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/4064534253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48213800 0x562f484fab00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x562f490902d0 tx=0x562f4855efc0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 reap_dead start 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3cd8c0700 1 -- 172.21.15.191:0/4064534253 reap_dead start 2024-04-27T04:35:06.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 shutdown_connections 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 >> 172.21.15.191:0/4064534253 conn(0x562f48212c00 msgr2=0x562f482ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 shutdown_connections 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/4064534253 wait complete. 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f48640c00 msgr2=0x562f486f4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x562f48640c00 0x562f486f4580 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f4908dc00 msgr2=0x562f486f4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x562f4908dc00 0x562f486f4000 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48644800 msgr2=0x562f486f5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x562f48644800 0x562f486f5080 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f48640400 msgr2=0x562f486f4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x562f48640400 0x562f486f4b00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48641c00 msgr2=0x562f484fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48641c00 0x562f484fd700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x562f485d5ef0 tx=0x562f4861d5c0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 msgr2=0x562f484fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/1357579474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4864fc00 0x562f484fc680 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x562f4865a810 tx=0x562f4861c5a0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3ce0c1700 1 -- 172.21.15.191:0/1357579474 reap_dead start 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 shutdown_connections 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 >> 172.21.15.191:0/1357579474 conn(0x562f4864f800 msgr2=0x562f48300000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 shutdown_connections 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.911+0000 7fa3d0117a80 1 -- 172.21.15.191:0/1357579474 wait complete. 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48213400 msgr2=0x562f484fb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48213400 0x562f484fb080 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x562f49090ae0 tx=0x562f4855ede0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 msgr2=0x562f484fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x562f48fd96b0 tx=0x562f48303a40 comp rx=0 tx=0).stop 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 shutdown_connections 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x562f48213400 0x562f484fb080 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 --2- 172.21.15.191:0/3113385927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f48212800 0x562f484fa580 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 >> 172.21.15.191:0/3113385927 conn(0x562f48212400 msgr2=0x562f482ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 shutdown_connections 2024-04-27T04:35:06.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.912+0000 7fa3d0117a80 1 -- 172.21.15.191:0/3113385927 wait complete. 2024-04-27T04:35:06.919 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T04:35:06.919 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T04:35:06.919 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-04-27T04:35:06.966 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:06.966 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.965+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cdf20a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55af66fe2c00 0x55af672ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cdf20a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55af6730d500 con 0x55af66fe2800 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cdf20a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55af67084a00 con 0x55af66fe2c00 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cb6c9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55af66fe2c00 0x55af672ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cbeca700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55af66fe2800 0x55af670ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57128/0 (socket says 172.21.15.191:57128) 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cbeca700 1 -- 172.21.15.191:0/3213058598 learned_addr learned my addr 172.21.15.191:0/3213058598 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:06.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3213058598 >> v1:172.21.15.138:6789/0 conn(0x55af66fe2800 legacy=0x55af670ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3213058598 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700bd40 con 0x55af66fe2c00 2024-04-27T04:35:06.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.966+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/3213058598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2c00 0x55af672ca580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55af672c5890 tx=0x55af670d3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8de4fad13405eb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69caec8700 1 -- 172.21.15.191:0/3213058598 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b4a0 con 0x55af66fe2c00 2024-04-27T04:35:06.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69caec8700 1 -- 172.21.15.191:0/3213058598 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af6730d800 con 0x55af66fe2c00 2024-04-27T04:35:06.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69caec8700 1 -- 172.21.15.191:0/3213058598 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b600 con 0x55af66fe2c00 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3213058598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2c00 msgr2=0x55af672ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3213058598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2c00 0x55af672ca580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55af672c5890 tx=0x55af670d3920 comp rx=0 tx=0).stop 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3213058598 shutdown_connections 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3213058598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2c00 0x55af672ca580 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3213058598 >> 172.21.15.191:0/3213058598 conn(0x55af66fe2400 msgr2=0x55af670ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3213058598 shutdown_connections 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.967+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3213058598 wait complete. 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:06.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cdf20a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cdf20a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af67084500 con 0x55af66fe2800 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54556/0 (socket says 172.21.15.191:54556) 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cbeca700 1 -- 172.21.15.191:0/3235550776 learned_addr learned my addr 172.21.15.191:0/3235550776 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.968+0000 7f69cbeca700 1 -- 172.21.15.191:0/3235550776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700bba0 con 0x55af66fe2800 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cbeca700 1 --2- 172.21.15.191:0/3235550776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55af67dad800 tx=0x55af670d3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0a8a0dbdb32eab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69ca6c7700 1 -- 172.21.15.191:0/3235550776 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b1e0 con 0x55af66fe2800 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69ca6c7700 1 -- 172.21.15.191:0/3235550776 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af6730db00 con 0x55af66fe2800 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69ca6c7700 1 -- 172.21.15.191:0/3235550776 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b4a0 con 0x55af66fe2800 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3235550776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 msgr2=0x55af672ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3235550776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55af67dad800 tx=0x55af670d3b60 comp rx=0 tx=0).stop 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3235550776 shutdown_connections 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3235550776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3235550776 >> 172.21.15.191:0/3235550776 conn(0x55af66fe2400 msgr2=0x55af670cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3235550776 shutdown_connections 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3235550776 wait complete. 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.969+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cdf20a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cdf20a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af670843c0 con 0x55af66fe2800 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54566/0 (socket says 172.21.15.191:54566) 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cbeca700 1 -- 172.21.15.191:0/571684288 learned_addr learned my addr 172.21.15.191:0/571684288 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cbeca700 1 -- 172.21.15.191:0/571684288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700ba00 con 0x55af66fe2800 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.970+0000 7f69cbeca700 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55af67dada40 tx=0x55af670d3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69c8ec4700 1 -- 172.21.15.191:0/571684288 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b600 con 0x55af66fe2800 2024-04-27T04:35:06.984 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69c8ec4700 1 -- 172.21.15.191:0/571684288 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af67083c80 con 0x55af66fe2800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af6700b860 con 0x55af66fe2800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69c8ec4700 1 -- 172.21.15.191:0/571684288 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b1e0 con 0x55af66fe2800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af6700b6c0 con 0x55af66fe2800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69c8ec4700 1 -- 172.21.15.191:0/571684288 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55af67d84b00 con 0x55af66fe2800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69c8ec4700 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af66fe3400 0x55af672cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69c8ec4700 1 -- 172.21.15.191:0/571684288 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55af6705d180 con 0x55af66fe2800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.971+0000 7f69cdf20a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cdf20a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6701e640 con 0x55af66fe3800 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af66fe3400 0x55af672cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.985 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54576/0 (socket says 172.21.15.191:54576) 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cbeca700 1 -- 172.21.15.191:0/1777322477 learned_addr learned my addr 172.21.15.191:0/1777322477 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af66fe3400 0x55af672cb080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55af67e76a20 tx=0x55af67332de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cbeca700 1 -- 172.21.15.191:0/1777322477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700a000 con 0x55af66fe3800 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69cbeca700 1 --2- 172.21.15.191:0/1777322477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55af67e76360 tx=0x55af67332240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846919058e315f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69c66bf700 1 -- 172.21.15.191:0/1777322477 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b4a0 con 0x55af66fe3800 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69c66bf700 1 -- 172.21.15.191:0/1777322477 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af6730c300 con 0x55af66fe3800 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.972+0000 7f69c66bf700 1 -- 172.21.15.191:0/1777322477 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b8c0 con 0x55af66fe3800 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- 172.21.15.191:0/1777322477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 msgr2=0x55af672cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/1777322477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55af67e76360 tx=0x55af67332240 comp rx=0 tx=0).stop 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- 172.21.15.191:0/1777322477 shutdown_connections 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/1777322477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:06.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- 172.21.15.191:0/1777322477 >> 172.21.15.191:0/1777322477 conn(0x55af66fe2c00 msgr2=0x55af670cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- 172.21.15.191:0/1777322477 shutdown_connections 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- 172.21.15.191:0/1777322477 wait complete. 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cdf20a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6701e8c0 con 0x55af66fe3800 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cbeca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54578/0 (socket says 172.21.15.191:54578) 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.973+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 learned_addr learned my addr 172.21.15.191:0/3363820519 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700aea0 con 0x55af66fe3800 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69cbeca700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55af67e76690 tx=0x55af67332fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69c4ebc700 1 -- 172.21.15.191:0/3363820519 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b4a0 con 0x55af66fe3800 2024-04-27T04:35:06.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69c4ebc700 1 -- 172.21.15.191:0/3363820519 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af67083e00 con 0x55af66fe3800 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af6700a820 con 0x55af66fe3800 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69c4ebc700 1 -- 172.21.15.191:0/3363820519 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b8c0 con 0x55af66fe3800 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af6700a9c0 con 0x55af66fe3800 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.974+0000 7f69c4ebc700 1 -- 172.21.15.191:0/3363820519 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55af67d84dc0 con 0x55af66fe3800 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69c4ebc700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6738ec00 0x55af672cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69c4ebc700 1 -- 172.21.15.191:0/3363820519 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55af6705d340 con 0x55af66fe3800 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6738ec00 0x55af672cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6738ec00 0x55af672cb600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55af672c5890 tx=0x55af670d3800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55af66fe3c00 0x55af672cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738f000 con 0x55af66fe3c00 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.975+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55af66fe3c00 0x55af672cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55af66fe3c00 0x55af672cbb80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 realm 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738f400 con 0x55af66fe3c00 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738f800 con 0x55af66fe3c00 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.976+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738fc00 con 0x55af66fe3c00 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55af6738f000 0x55af672cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738f400 con 0x55af6738f000 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cbeca700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55af6738f000 0x55af672cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cbeca700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55af6738f000 0x55af672cc100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 20 searching for the correct realm 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.977+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c000 con 0x55af6738f000 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:06.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cbeca700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55af6740c000 0x55af672ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738f400 con 0x55af6740c000 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55af6740c000 0x55af672ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55af6740c000 0x55af672ccc00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55af6738f400 0x55af672cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.978+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55af6738f400 0x55af672cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55af6738f400 0x55af672cd180 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:06.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.979+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6740c000 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.980+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:06.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6740c000 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.981+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:07.011 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.982+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6740c000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.983+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6740c000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.984+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6740c000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f400 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af66fe3c00 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.985+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6740c000 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:07.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c400 con 0x55af66fe3c00 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c800 con 0x55af66fe3c00 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740cc00 con 0x55af6738f000 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 zone default found 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 4 Realm: () 2024-04-27T04:35:07.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 zonegroup default 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.986+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d000 con 0x55af6738f400 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d400 con 0x55af6738f000 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c400 con 0x55af6738f000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 started sync module instance, tier type = 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.987+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c800 con 0x55af6740c000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.988+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.988+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94211838581504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d000 con 0x55af6740c000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.988+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740dc00 con 0x55af66fe3c00 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.989+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94211838581504] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.989+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.989+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94211838582656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738fc00 con 0x55af66fe3c00 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.989+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738f800 con 0x55af6738f000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.990+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.990+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94211838583808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d800 con 0x55af6738f000 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.990+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c800 con 0x55af6740c000 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.991+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94211838582656] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.991+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94211838583808] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.992+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.992+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94211838584960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d000 con 0x55af6740c000 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.992+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740cc00 con 0x55af66fe3c00 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.992+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94211838584960] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.993+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.993+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94211838586112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d400 con 0x55af66fe3c00 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.993+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c400 con 0x55af6738f400 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.994+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.994+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94211838586112] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.994+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94211838587264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738e400 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.994+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418000 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.994+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(56 notify.6 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.994+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94211838588416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418400 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.995+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418800 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.995+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94211838587264] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.995+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94211838588416] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.995+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94211838589568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418c00 con 0x55af6738f400 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=0 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=1 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=2 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=3 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=4 2024-04-27T04:35:07.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=5 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=6 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94211838589568] v33'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 20 add_watcher() i=7 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.996+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67419000 con 0x55af66fe3c00 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.997+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.997+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94211840016384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67419400 con 0x55af66fe3c00 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.998+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94211840016384] v33'193 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.998+0000 7f69b2e98700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.998+0000 7f69cdf20a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.998+0000 7f69cdf20a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.998+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.998+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:07.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cdf20a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cdf20a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af670843c0 con 0x55af67419c00 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cb6c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cb6c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54588/0 (socket says 172.21.15.191:54588) 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cb6c9700 1 -- 172.21.15.191:0/4290661643 learned_addr learned my addr 172.21.15.191:0/4290661643 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cb6c9700 1 -- 172.21.15.191:0/4290661643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700ab60 con 0x55af67419c00 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/4290661643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55af67426660 tx=0x55af670d3980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1701c80291a66b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:06.999+0000 7f69b2697700 1 -- 172.21.15.191:0/4290661643 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6709b760 con 0x55af67419c00 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69b2697700 1 -- 172.21.15.191:0/4290661643 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af67436000 con 0x55af67419c00 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69b2697700 1 -- 172.21.15.191:0/4290661643 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6741e160 con 0x55af67419c00 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 -- 172.21.15.191:0/4290661643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 msgr2=0x55af672cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/4290661643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55af67426660 tx=0x55af670d3980 comp rx=0 tx=0).stop 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 -- 172.21.15.191:0/4290661643 shutdown_connections 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/4290661643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 -- 172.21.15.191:0/4290661643 >> 172.21.15.191:0/4290661643 conn(0x55af67419800 msgr2=0x55af670d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 -- 172.21.15.191:0/4290661643 shutdown_connections 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.000+0000 7f69cdf20a80 1 -- 172.21.15.191:0/4290661643 wait complete. 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cdf20a80 1 Processor -- start 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cdf20a80 1 -- start start 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cdf20a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cdf20a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af67084500 con 0x55af67419c00 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cb6c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cb6c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54604/0 (socket says 172.21.15.191:54604) 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cb6c9700 1 -- 172.21.15.191:0/2504963115 learned_addr learned my addr 172.21.15.191:0/2504963115 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cb6c9700 1 -- 172.21.15.191:0/2504963115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6700ad00 con 0x55af67419c00 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69cb6c9700 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55af67426870 tx=0x55af673ea5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.001+0000 7f69b0e94700 1 -- 172.21.15.191:0/2504963115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6741e2c0 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69b0e94700 1 -- 172.21.15.191:0/2504963115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af67436180 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af6700a340 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69b0e94700 1 -- 172.21.15.191:0/2504963115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af6741e420 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af6700bd40 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69b0e94700 1 -- 172.21.15.191:0/2504963115 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55af67d85b80 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69b0e94700 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6740dc00 0x55af672cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.002+0000 7f69b0e94700 1 -- 172.21.15.191:0/2504963115 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55af6705d6c0 con 0x55af67419c00 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.003+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6740dc00 0x55af672cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.003+0000 7f69cdf20a80 5 note: GC not initialized 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.003+0000 7f69cc6cb700 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6740dc00 0x55af672cd700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55af673abf80 tx=0x55af673eb5c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.003+0000 7f69ae68f700 20 reqs_thread_entry: start 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69ac68b700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:37 2024 2024-04-27T04:35:07.023 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69abe69700 20 reqs_thread_entry: start 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738e400 con 0x55af66fe3c00 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.004+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~369 in=369b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740d400 con 0x55af66fe3c00 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~369] v33'14 uv14 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cdf20a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:35:07.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cdf20a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cdf20a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94211840018688 in=557b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6740c400 con 0x55af66fe3c00 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94211838586112 notify 141733920784 ret 0) v3 ==== 587+0+0 (crc 0 0 0) 0x55af67518000 con 0x55af66fe3c00 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69c66bf700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920784 cookie 94211838586112 notifier 4708 bl.length()=545 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94211840018688 out=8b] v0'0 uv180 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69c66bf700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69c66bf700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69c66bf700 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418000 con 0x55af66fe3c00 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.006+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94211840018688 notify 141733920784 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55af675181a0 con 0x55af66fe3c00 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:35:07.026 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [ 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "user", 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "read" 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:07.027 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:35:07.028 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:35:07.029 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:07.029 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:35:07.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.007+0000 7f69b2e98700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:07.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.007+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94211840016384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af6738e400 con 0x55af66fe3c00 2024-04-27T04:35:07.034 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.008+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94211840016384] v33'194 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.008+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94211838581504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418800 con 0x55af6740c000 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.009+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94211838581504] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.009+0000 7f69cdf20a80 20 remove_watcher() i=0 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.009+0000 7f69cdf20a80 2 removed watcher, disabling cache 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.009+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94211838582656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67418c00 con 0x55af66fe3c00 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.010+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94211838582656] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.010+0000 7f69cdf20a80 20 remove_watcher() i=1 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.010+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94211838583808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67419000 con 0x55af6738f000 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.011+0000 7f69cbeca700 1 -- 172.21.15.191:0/3363820519 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94211838583808] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fcfc0 con 0x55af6738f000 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.011+0000 7f69cdf20a80 20 remove_watcher() i=2 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.011+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94211838584960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af67419400 con 0x55af6740c000 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.012+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94211838584960] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fd200 con 0x55af6740c000 2024-04-27T04:35:07.035 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.012+0000 7f69cdf20a80 20 remove_watcher() i=3 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.012+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94211838586112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af673f2000 con 0x55af66fe3c00 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.013+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94211838586112] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af66fe3c00 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.013+0000 7f69cdf20a80 20 remove_watcher() i=4 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.013+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94211838587264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af673f2400 con 0x55af6738f400 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.014+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94211838587264] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.014+0000 7f69cdf20a80 20 remove_watcher() i=5 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.014+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94211838588416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af673f2800 con 0x55af6738f400 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.015+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94211838588416] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.015+0000 7f69cdf20a80 20 remove_watcher() i=6 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.015+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94211838589568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55af673f2c00 con 0x55af6738f400 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cc6cb700 1 -- 172.21.15.191:0/3363820519 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94211838589568] v33'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af673fc900 con 0x55af6738f400 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 20 remove_watcher() i=7 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55af6738f400 msgr2=0x55af672cd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55af6738f400 0x55af672cd180 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55af66fe3c00 msgr2=0x55af672cbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55af66fe3c00 0x55af672cbb80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55af6740c000 msgr2=0x55af672ccc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55af6740c000 0x55af672ccc00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55af6738f000 msgr2=0x55af672cc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55af6738f000 0x55af672cc100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6738ec00 msgr2=0x55af672cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6738ec00 0x55af672cb600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55af672c5890 tx=0x55af670d3800 comp rx=0 tx=0).stop 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 msgr2=0x55af672cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/3363820519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe3800 0x55af672cab00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55af67e76690 tx=0x55af67332fc0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 reap_dead start 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cb6c9700 1 -- 172.21.15.191:0/3363820519 reap_dead start 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 shutdown_connections 2024-04-27T04:35:07.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 >> 172.21.15.191:0/3363820519 conn(0x55af66fe2c00 msgr2=0x55af670cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 shutdown_connections 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.016+0000 7f69cdf20a80 1 -- 172.21.15.191:0/3363820519 wait complete. 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6740dc00 msgr2=0x55af672cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6740dc00 0x55af672cd700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55af673abf80 tx=0x55af673eb5c0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 msgr2=0x55af672cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55af67426870 tx=0x55af673ea5a0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 shutdown_connections 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af6740dc00 0x55af672cd700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/2504963115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af67419c00 0x55af672cc680 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 >> 172.21.15.191:0/2504963115 conn(0x55af67419800 msgr2=0x55af670d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 shutdown_connections 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/2504963115 wait complete. 2024-04-27T04:35:07.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af66fe3400 msgr2=0x55af672cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af66fe3400 0x55af672cb080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55af67e76a20 tx=0x55af67332de0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 msgr2=0x55af672ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55af67dada40 tx=0x55af670d3a40 comp rx=0 tx=0).stop 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 shutdown_connections 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55af66fe3400 0x55af672cb080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 --2- 172.21.15.191:0/571684288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af66fe2800 0x55af672ca580 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 >> 172.21.15.191:0/571684288 conn(0x55af66fe2400 msgr2=0x55af670cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 shutdown_connections 2024-04-27T04:35:07.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.017+0000 7f69cdf20a80 1 -- 172.21.15.191:0/571684288 wait complete. 2024-04-27T04:35:07.039 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:35:07.039 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T04:35:07.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T04:35:07.040 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-04-27T04:35:07.087 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:07.087 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:07.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.088 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff8e88a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff8e88a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cab312b500 con 0x55cab2e00c00 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff8e88a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cab2ea2a00 con 0x55cab2e00800 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff6e32700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff6e32700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54614/0 (socket says 172.21.15.191:54614) 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff6631700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55cab2e00c00 0x55cab2eec800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57138/0 (socket says 172.21.15.191:57138) 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.086+0000 7f0ff6e32700 1 -- 172.21.15.191:0/2372348672 learned_addr learned my addr 172.21.15.191:0/2372348672 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff6e32700 1 -- 172.21.15.191:0/2372348672 >> v1:172.21.15.138:6789/0 conn(0x55cab2e00c00 legacy=0x55cab2eec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff6e32700 1 -- 172.21.15.191:0/2372348672 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e29d40 con 0x55cab2e00800 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff5e30700 1 -- 172.21.15.191:0/2372348672 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190784976 0 0) 0x55cab3bce000 con 0x55cab2e00c00 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/2372348672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55cab30e39e0 tx=0x55cab2ef1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba004703e86e5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff5e30700 1 -- 172.21.15.191:0/2372348672 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb94a0 con 0x55cab2e00800 2024-04-27T04:35:07.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff5e30700 1 -- 172.21.15.191:0/2372348672 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab312b800 con 0x55cab2e00800 2024-04-27T04:35:07.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.087+0000 7f0ff5e30700 1 -- 172.21.15.191:0/2372348672 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb9600 con 0x55cab2e00800 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/2372348672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 msgr2=0x55cab30e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/2372348672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55cab30e39e0 tx=0x55cab2ef1920 comp rx=0 tx=0).stop 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/2372348672 shutdown_connections 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/2372348672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/2372348672 >> 172.21.15.191:0/2372348672 conn(0x55cab2e00400 msgr2=0x55cab2eec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/2372348672 shutdown_connections 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/2372348672 wait complete. 2024-04-27T04:35:07.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.088+0000 7f0ff8e88a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab2ea2500 con 0x55cab2e00800 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff6e32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff6e32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54620/0 (socket says 172.21.15.191:54620) 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3564934487 learned_addr learned my addr 172.21.15.191:0/3564934487 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3564934487 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e29ba0 con 0x55cab2e00800 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3564934487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55cab30e3f50 tx=0x55cab2ef1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9e53b0689b9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff562f700 1 -- 172.21.15.191:0/3564934487 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb9760 con 0x55cab2e00800 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff562f700 1 -- 172.21.15.191:0/3564934487 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab312bb00 con 0x55cab2e00800 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff562f700 1 -- 172.21.15.191:0/3564934487 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb94a0 con 0x55cab2e00800 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3564934487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 msgr2=0x55cab30e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3564934487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55cab30e3f50 tx=0x55cab2ef1800 comp rx=0 tx=0).stop 2024-04-27T04:35:07.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3564934487 shutdown_connections 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3564934487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3564934487 >> 172.21.15.191:0/3564934487 conn(0x55cab2e00400 msgr2=0x55cab2eed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3564934487 shutdown_connections 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.089+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3564934487 wait complete. 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff8e88a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff8e88a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab2ea23c0 con 0x55cab2e00800 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff6e32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff6e32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54634/0 (socket says 172.21.15.191:54634) 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3455253762 learned_addr learned my addr 172.21.15.191:0/3455253762 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3455253762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e29a00 con 0x55cab2e00800 2024-04-27T04:35:07.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.090+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55cab3bc7950 tx=0x55cab2ef1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff3e2c700 1 -- 172.21.15.191:0/3455253762 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb9600 con 0x55cab2e00800 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cab2e29860 con 0x55cab2e00800 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff3e2c700 1 -- 172.21.15.191:0/3455253762 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab312bc80 con 0x55cab2e00800 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff3e2c700 1 -- 172.21.15.191:0/3455253762 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb98c0 con 0x55cab2e00800 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cab2e296c0 con 0x55cab2e00800 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff3e2c700 1 -- 172.21.15.191:0/3455253762 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cab3ba2b00 con 0x55cab2e00800 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.091+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff8e88a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff8e88a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab2e3c640 con 0x55cab2e01400 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff3e2c700 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318a000 0x55cab30e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff3e2c700 1 -- 172.21.15.191:0/3455253762 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55cab2e7b180 con 0x55cab2e00800 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318a000 0x55cab30e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff7633700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff7633700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54640/0 (socket says 172.21.15.191:54640) 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff7633700 1 -- 172.21.15.191:0/3940191603 learned_addr learned my addr 172.21.15.191:0/3940191603 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff7633700 1 -- 172.21.15.191:0/3940191603 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e28000 con 0x55cab2e01400 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318a000 0x55cab30e9080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55cab3c88ed0 tx=0x55cab314eb40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff7633700 1 --2- 172.21.15.191:0/3940191603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55cab3c88210 tx=0x55cab314ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c7279834d7e75b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff1627700 1 -- 172.21.15.191:0/3940191603 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb9600 con 0x55cab2e01400 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff1627700 1 -- 172.21.15.191:0/3940191603 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab2ea1c80 con 0x55cab2e01400 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.092+0000 7f0ff1627700 1 -- 172.21.15.191:0/3940191603 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb98c0 con 0x55cab2e01400 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3940191603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 msgr2=0x55cab30e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3940191603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55cab3c88210 tx=0x55cab314ee40 comp rx=0 tx=0).stop 2024-04-27T04:35:07.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3940191603 shutdown_connections 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3940191603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3940191603 >> 172.21.15.191:0/3940191603 conn(0x55cab2e00c00 msgr2=0x55cab2eed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3940191603 shutdown_connections 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3940191603 wait complete. 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff8e88a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab2e3c8c0 con 0x55cab2e01400 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff7633700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff7633700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54652/0 (socket says 172.21.15.191:54652) 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 learned_addr learned my addr 172.21.15.191:0/3890660057 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.093+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e28ea0 con 0x55cab2e01400 2024-04-27T04:35:07.116 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0ff7633700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55cab3c885a0 tx=0x55cab314ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0fefe24700 1 -- 172.21.15.191:0/3890660057 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb9760 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0fefe24700 1 -- 172.21.15.191:0/3890660057 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab2ea1b00 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0fefe24700 1 -- 172.21.15.191:0/3890660057 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab2eb9600 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cab2e28820 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cab2e289c0 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.094+0000 7f0fefe24700 1 -- 172.21.15.191:0/3890660057 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cab3ba2dc0 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0fefe24700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318ac00 0x55cab30e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0fefe24700 1 -- 172.21.15.191:0/3890660057 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55cab2e7b340 con 0x55cab2e01400 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff6631700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318ac00 0x55cab30e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cab2e01800 0x55cab30e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab2e01c00 con 0x55cab2e01800 2024-04-27T04:35:07.117 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff7633700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cab2e01800 0x55cab30e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff6631700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318ac00 0x55cab30e9600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55cab3189cb0 tx=0x55cab32400c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff7633700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cab2e01800 0x55cab30e9b80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 20 realm 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.095+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b000 con 0x55cab2e01800 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b400 con 0x55cab2e01800 2024-04-27T04:35:07.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b800 con 0x55cab2e01800 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cab318bc00 0x55cab30ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b000 con 0x55cab318bc00 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.096+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cab318bc00 0x55cab30ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cab318bc00 0x55cab30ea100 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff8e88a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff8e88a80 20 searching for the correct realm 2024-04-27T04:35:07.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b400 con 0x55cab318bc00 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff6e32700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cab318b400 0x55cab30eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b000 con 0x55cab318b400 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.097+0000 7f0ff6631700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cab318b400 0x55cab30eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.098+0000 7f0ff6631700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cab318b400 0x55cab30eac00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.098+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.098+0000 7f0ff6631700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cab318b000 0x55cab30eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.098+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.098+0000 7f0ff7633700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cab318b000 0x55cab30eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.098+0000 7f0ff7633700 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cab318b000 0x55cab30eb180 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.120 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.099+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b400 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.100+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b400 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.101+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.102+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b400 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.103+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b400 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b400 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318bc00 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.104+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.124 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b400 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab2e01c00 con 0x55cab2e01800 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.105+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252400 con 0x55cab2e01800 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.125 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252800 con 0x55cab318bc00 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 zone default found 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 4 Realm: () 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 zonegroup default 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252c00 con 0x55cab318b000 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.126 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253000 con 0x55cab318bc00 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.106+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253400 con 0x55cab318bc00 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.107+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.107+0000 7f0ff8e88a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.107+0000 7f0ff8e88a80 20 started sync module instance, tier type = 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.107+0000 7f0ff8e88a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.107+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253800 con 0x55cab318b400 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.108+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.108+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94329075792640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252800 con 0x55cab318b400 2024-04-27T04:35:07.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.108+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253000 con 0x55cab2e01800 2024-04-27T04:35:07.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.109+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94329075792640] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.109+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.109+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94329075793792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253400 con 0x55cab2e01800 2024-04-27T04:35:07.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.109+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252400 con 0x55cab318bc00 2024-04-27T04:35:07.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.110+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.110+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94329075794944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252c00 con 0x55cab318bc00 2024-04-27T04:35:07.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.110+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab2e01c00 con 0x55cab318b400 2024-04-27T04:35:07.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.110+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94329075793792] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.112+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.112+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94329075794944] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.112+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94329075796096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab318b800 con 0x55cab318b400 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.113+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab2e01800 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.114+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.114+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94329075797248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253800 con 0x55cab2e01800 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.114+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252800 con 0x55cab318b000 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.114+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94329075796096] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.115+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94329075797248] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.115+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.115+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94329075798400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253c00 con 0x55cab318b000 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.115+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253000 con 0x55cab318b000 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.116+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94329075798400] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.116+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.116+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94329075799552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253400 con 0x55cab318b000 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.116+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252000 con 0x55cab318b000 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(58 notify.7 [create] v33'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94329075799552] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94329075800704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253800 con 0x55cab318b000 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=0 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=1 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=2 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=3 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=4 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=5 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.117+0000 7f0ff8e88a80 20 add_watcher() i=6 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.118+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94329075800704] v33'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.118+0000 7f0ff8e88a80 20 add_watcher() i=7 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.118+0000 7f0ff8e88a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.118+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3252800 con 0x55cab2e01800 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.118+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.118+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94329077358592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253c00 con 0x55cab2e01800 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94329077358592] v33'195 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0fdde00700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff8e88a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff8e88a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff8e88a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.119+0000 7f0ff8e88a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab2ea23c0 con 0x55cab3253400 2024-04-27T04:35:07.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0ff6631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0ff6631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54668/0 (socket says 172.21.15.191:54668) 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0ff6631700 1 -- 172.21.15.191:0/1618101094 learned_addr learned my addr 172.21.15.191:0/1618101094 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0ff6631700 1 -- 172.21.15.191:0/1618101094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e28b60 con 0x55cab3253400 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0ff6631700 1 --2- 172.21.15.191:0/1618101094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55cab31f55f0 tx=0x55cab314ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7636e3624272cae server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0fdd5ff700 1 -- 172.21.15.191:0/1618101094 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab3270000 con 0x55cab3253400 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0fdd5ff700 1 -- 172.21.15.191:0/1618101094 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab3274000 con 0x55cab3253400 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.120+0000 7f0fdd5ff700 1 -- 172.21.15.191:0/1618101094 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab3270160 con 0x55cab3253400 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/1618101094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 msgr2=0x55cab30ea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/1618101094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55cab31f55f0 tx=0x55cab314ff80 comp rx=0 tx=0).stop 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/1618101094 shutdown_connections 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/1618101094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/1618101094 >> 172.21.15.191:0/1618101094 conn(0x55cab3253000 msgr2=0x55cab2eee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/1618101094 shutdown_connections 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/1618101094 wait complete. 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 Processor -- start 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- start start 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.121+0000 7f0ff8e88a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab2ea2500 con 0x55cab3253400 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff6631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff6631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54672/0 (socket says 172.21.15.191:54672) 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff6631700 1 -- 172.21.15.191:0/352932934 learned_addr learned my addr 172.21.15.191:0/352932934 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff6631700 1 -- 172.21.15.191:0/352932934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab2e28d00 con 0x55cab3253400 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff6631700 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55cab3272210 tx=0x55cab32404e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0fdbdfc700 1 -- 172.21.15.191:0/352932934 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab32702c0 con 0x55cab3253400 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0fdbdfc700 1 -- 172.21.15.191:0/352932934 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cab3274180 con 0x55cab3253400 2024-04-27T04:35:07.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0fdbdfc700 1 -- 172.21.15.191:0/352932934 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cab3270000 con 0x55cab3253400 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cab2e28340 con 0x55cab3253400 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.122+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cab2e28b60 con 0x55cab3253400 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.123+0000 7f0fdbdfc700 1 -- 172.21.15.191:0/352932934 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55cab3ba3b80 con 0x55cab3253400 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.123+0000 7f0fdbdfc700 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab32f4000 0x55cab30eb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.123+0000 7f0fdbdfc700 1 -- 172.21.15.191:0/352932934 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55cab2e7b500 con 0x55cab3253400 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.123+0000 7f0ff8e88a80 5 note: GC not initialized 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.123+0000 7f0ff7633700 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab32f4000 0x55cab30eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.123+0000 7f0ff7633700 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab32f4000 0x55cab30eb700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55cab31891a0 tx=0x55cab3240360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0fd95f7700 20 reqs_thread_entry: start 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0ff8e88a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0ff8e88a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0fd75f3700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:37 2024 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:07.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0fd6dd1700 20 reqs_thread_entry: start 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0ff8e88a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0ff8e88a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.124+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3253800 con 0x55cab2e01800 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.125+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~369 out=369b,getxattrs out=75b] v0'0 uv14 ondisk = 0) v8 ==== 231+0+492 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.125+0000 7f0ff8e88a80 20 rados_obj.operate() r=0 bl.length=369 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.125+0000 7f0ff8e88a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.125+0000 7f0ff8e88a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.125+0000 7f0ff8e88a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.125+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f4400 con 0x55cab2e01800 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v33'15 uv15 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff8e88a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff8e88a80 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff8e88a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:35:07.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94329077360896 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f4800 con 0x55cab2e01800 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== watch-notify(notify (1) cookie 94329075797248 notify 141733920785 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55cab31f3d40 con 0x55cab2e01800 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff1627700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920785 cookie 94329075797248 notifier 4716 bl.length()=533 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(64 notify.4 [notify cookie 94329077360896 out=8b] v0'0 uv183 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff1627700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff1627700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.126+0000 7f0ff1627700 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3366000 con 0x55cab2e01800 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.127+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.127+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== watch-notify(notify_complete (2) cookie 94329077360896 notify 141733920785 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55cab31f3a00 con 0x55cab2e01800 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:07.145 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "foo", 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Foo", 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "foo@foo.com", 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:07.146 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:35:07.147 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:07.148 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.127+0000 7f0fdde00700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.127+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94329077358592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f4c00 con 0x55cab2e01800 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.128+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94329077358592] v33'196 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.128+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94329075792640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f5000 con 0x55cab318b400 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.129+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94329075792640] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.129+0000 7f0ff8e88a80 20 remove_watcher() i=0 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.129+0000 7f0ff8e88a80 2 removed watcher, disabling cache 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.129+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94329075793792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f5400 con 0x55cab2e01800 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.131+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94329075793792] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.131+0000 7f0ff8e88a80 20 remove_watcher() i=1 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.131+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94329075794944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f5800 con 0x55cab318bc00 2024-04-27T04:35:07.154 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.132+0000 7f0ff6e32700 1 -- 172.21.15.191:0/3890660057 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94329075794944] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245200 con 0x55cab318bc00 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.132+0000 7f0ff8e88a80 20 remove_watcher() i=2 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.132+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94329075796096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab32f5c00 con 0x55cab318b400 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.133+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94329075796096] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3245440 con 0x55cab318b400 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.133+0000 7f0ff8e88a80 20 remove_watcher() i=3 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.133+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94329075797248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3366000 con 0x55cab2e01800 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.134+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94329075797248] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab2e01800 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.134+0000 7f0ff8e88a80 20 remove_watcher() i=4 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.134+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94329075798400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3366800 con 0x55cab318b000 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.135+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94329075798400] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.135+0000 7f0ff8e88a80 20 remove_watcher() i=5 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.135+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94329075799552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3366c00 con 0x55cab318b000 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.136+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94329075799552] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.136+0000 7f0ff8e88a80 20 remove_watcher() i=6 2024-04-27T04:35:07.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.136+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94329075800704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cab3367000 con 0x55cab318b000 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff7633700 1 -- 172.21.15.191:0/3890660057 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94329075800704] v33'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cab3244b40 con 0x55cab318b000 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 20 remove_watcher() i=7 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cab318b000 msgr2=0x55cab30eb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55cab318b000 0x55cab30eb180 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cab2e01800 msgr2=0x55cab30e9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55cab2e01800 0x55cab30e9b80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cab318b400 msgr2=0x55cab30eac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55cab318b400 0x55cab30eac00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cab318bc00 msgr2=0x55cab30ea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55cab318bc00 0x55cab30ea100 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318ac00 msgr2=0x55cab30e9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318ac00 0x55cab30e9600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55cab3189cb0 tx=0x55cab32400c0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 msgr2=0x55cab30e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3890660057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e01400 0x55cab30e8b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55cab3c885a0 tx=0x55cab314ea20 comp rx=0 tx=0).stop 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 reap_dead start 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff6631700 1 -- 172.21.15.191:0/3890660057 reap_dead start 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 shutdown_connections 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 >> 172.21.15.191:0/3890660057 conn(0x55cab2e00c00 msgr2=0x55cab2eed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 shutdown_connections 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.137+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3890660057 wait complete. 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab32f4000 msgr2=0x55cab30eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab32f4000 0x55cab30eb700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55cab31891a0 tx=0x55cab3240360 comp rx=0 tx=0).stop 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 msgr2=0x55cab30ea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55cab3272210 tx=0x55cab32404e0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 shutdown_connections 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab32f4000 0x55cab30eb700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/352932934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab3253400 0x55cab30ea680 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 >> 172.21.15.191:0/352932934 conn(0x55cab3253000 msgr2=0x55cab2eee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 shutdown_connections 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/352932934 wait complete. 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318a000 msgr2=0x55cab30e9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318a000 0x55cab30e9080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55cab3c88ed0 tx=0x55cab314eb40 comp rx=0 tx=0).stop 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 msgr2=0x55cab30e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55cab3bc7950 tx=0x55cab2ef1920 comp rx=0 tx=0).stop 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 shutdown_connections 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55cab318a000 0x55cab30e9080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 --2- 172.21.15.191:0/3455253762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cab2e00800 0x55cab30e8580 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 >> 172.21.15.191:0/3455253762 conn(0x55cab2e00400 msgr2=0x55cab2eed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 shutdown_connections 2024-04-27T04:35:07.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.138+0000 7f0ff8e88a80 1 -- 172.21.15.191:0/3455253762 wait complete. 2024-04-27T04:35:07.159 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:35:07.174 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:07.174 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T04:35:07.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T04:35:07.174 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T04:35:07.219 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:07.219 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc541eb2a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc541eb2a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5645b2ee3500 con 0x5645b2bb8c00 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc541eb2a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5645b2c5aa00 con 0x5645b2bb8800 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc53fe5c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc53f65b700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5645b2bb8c00 0x5645b2ca4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:57142/0 (socket says 172.21.15.191:57142) 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc53f65b700 1 -- 172.21.15.191:0/2199763693 learned_addr learned my addr 172.21.15.191:0/2199763693 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc53fe5c700 1 -- 172.21.15.191:0/2199763693 >> v1:172.21.15.138:6789/0 conn(0x5645b2bb8c00 legacy=0x5645b2ca4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc53fe5c700 1 -- 172.21.15.191:0/2199763693 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be1d40 con 0x5645b2bb8800 2024-04-27T04:35:07.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.219+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/2199763693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5645b2e9b9e0 tx=0x5645b2ca9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed4624477afd35a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc53ee5a700 1 -- 172.21.15.191:0/2199763693 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c714a0 con 0x5645b2bb8800 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc53ee5a700 1 -- 172.21.15.191:0/2199763693 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b2ee3800 con 0x5645b2bb8800 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc53ee5a700 1 -- 172.21.15.191:0/2199763693 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c71600 con 0x5645b2bb8800 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2199763693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 msgr2=0x5645b2ea0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/2199763693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5645b2e9b9e0 tx=0x5645b2ca9920 comp rx=0 tx=0).stop 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2199763693 shutdown_connections 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/2199763693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2199763693 >> 172.21.15.191:0/2199763693 conn(0x5645b2bb8400 msgr2=0x5645b2ca4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2199763693 shutdown_connections 2024-04-27T04:35:07.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.220+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2199763693 wait complete. 2024-04-27T04:35:07.243 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc541eb2a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc541eb2a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2c5a500 con 0x5645b2bb8800 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54692/0 (socket says 172.21.15.191:54692) 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc53fe5c700 1 -- 172.21.15.191:0/1579227675 learned_addr learned my addr 172.21.15.191:0/1579227675 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc53fe5c700 1 -- 172.21.15.191:0/1579227675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be1ba0 con 0x5645b2bb8800 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.221+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/1579227675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5645b3a23800 tx=0x5645b2ca9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9433d1ed5d3ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc53e659700 1 -- 172.21.15.191:0/1579227675 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c71760 con 0x5645b2bb8800 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc53e659700 1 -- 172.21.15.191:0/1579227675 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b2ee3b00 con 0x5645b2bb8800 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc53e659700 1 -- 172.21.15.191:0/1579227675 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c714a0 con 0x5645b2bb8800 2024-04-27T04:35:07.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1579227675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 msgr2=0x5645b2ea0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/1579227675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5645b3a23800 tx=0x5645b2ca9800 comp rx=0 tx=0).stop 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1579227675 shutdown_connections 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/1579227675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1579227675 >> 172.21.15.191:0/1579227675 conn(0x5645b2bb8400 msgr2=0x5645b2ca5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1579227675 shutdown_connections 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1579227675 wait complete. 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.222+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc541eb2a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc541eb2a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2c5a3c0 con 0x5645b2bb8800 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54700/0 (socket says 172.21.15.191:54700) 2024-04-27T04:35:07.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53fe5c700 1 -- 172.21.15.191:0/1699907241 learned_addr learned my addr 172.21.15.191:0/1699907241 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53fe5c700 1 -- 172.21.15.191:0/1699907241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be1a00 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5645b3a23ec0 tx=0x5645b2ca9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53ce56700 1 -- 172.21.15.191:0/1699907241 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c71600 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53ce56700 1 -- 172.21.15.191:0/1699907241 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b2ee3c80 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc53ce56700 1 -- 172.21.15.191:0/1699907241 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c718c0 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645b2be1860 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.223+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b2be16c0 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc53ce56700 1 -- 172.21.15.191:0/1699907241 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5645b395ab00 con 0x5645b2bb8800 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc53ce56700 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2bb9400 0x5645b2ea1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.246 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc53ce56700 1 -- 172.21.15.191:0/1699907241 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5645b2c33180 con 0x5645b2bb8800 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc541eb2a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc541eb2a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2bf4640 con 0x5645b2bb9800 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc54065d700 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2bb9400 0x5645b2ea1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54714/0 (socket says 172.21.15.191:54714) 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.224+0000 7fc53fe5c700 1 -- 172.21.15.191:0/2203885922 learned_addr learned my addr 172.21.15.191:0/2203885922 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc54065d700 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2bb9400 0x5645b2ea1080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5645b2f40030 tx=0x5645b2f06b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc53fe5c700 1 -- 172.21.15.191:0/2203885922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be0000 con 0x5645b2bb9800 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/2203885922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5645b3a31050 tx=0x5645b2f06c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ff2303a2ad3a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc53a651700 1 -- 172.21.15.191:0/2203885922 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c71600 con 0x5645b2bb9800 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc53a651700 1 -- 172.21.15.191:0/2203885922 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b2ee3c80 con 0x5645b2bb9800 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc53a651700 1 -- 172.21.15.191:0/2203885922 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c718c0 con 0x5645b2bb9800 2024-04-27T04:35:07.247 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2203885922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 msgr2=0x5645b2ea0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/2203885922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5645b3a31050 tx=0x5645b2f06c60 comp rx=0 tx=0).stop 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2203885922 shutdown_connections 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/2203885922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2203885922 >> 172.21.15.191:0/2203885922 conn(0x5645b2bb8c00 msgr2=0x5645b2ca5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2203885922 shutdown_connections 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.225+0000 7fc541eb2a80 1 -- 172.21.15.191:0/2203885922 wait complete. 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc541eb2a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc541eb2a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2bf48c0 con 0x5645b2bb9800 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc53fe5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54730/0 (socket says 172.21.15.191:54730) 2024-04-27T04:35:07.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 learned_addr learned my addr 172.21.15.191:0/4047299336 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be0ea0 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5645b2f40600 tx=0x5645b2f06e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc538e4e700 1 -- 172.21.15.191:0/4047299336 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c71760 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.226+0000 7fc538e4e700 1 -- 172.21.15.191:0/4047299336 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b2c59e00 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645b2be0820 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc538e4e700 1 -- 172.21.15.191:0/4047299336 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2c714a0 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b2be09c0 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc538e4e700 1 -- 172.21.15.191:0/4047299336 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5645b395adc0 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc538e4e700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2f60c00 0x5645b2ea1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc538e4e700 1 -- 172.21.15.191:0/4047299336 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5645b2c33340 con 0x5645b2bb9800 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.227+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2f60c00 0x5645b2ea1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2bb9c00 0x5645b2ea1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61000 con 0x5645b2bb9c00 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc54065d700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2bb9c00 0x5645b2ea1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2f60c00 0x5645b2ea1600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5645b3a304e0 tx=0x5645b2fde000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc54065d700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2bb9c00 0x5645b2ea1b80 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 20 realm 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61400 con 0x5645b2bb9c00 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.228+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61800 con 0x5645b2bb9c00 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61c00 con 0x5645b2bb9c00 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2f61000 0x5645b2ea2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61400 con 0x5645b2f61000 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2f61000 0x5645b2ea2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2f61000 0x5645b2ea2100 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.229+0000 7fc541eb2a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc541eb2a80 20 searching for the correct realm 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea000 con 0x5645b2f61000 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b2fea000 0x5645b2ea2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61400 con 0x5645b2fea000 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b2fea000 0x5645b2ea2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b2fea000 0x5645b2ea2c00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2f61400 0x5645b2ea3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.230+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.231+0000 7fc54065d700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2f61400 0x5645b2ea3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.231+0000 7fc54065d700 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2f61400 0x5645b2ea3180 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.231+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.231+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.232+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.233+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.234+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.235+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.236+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61400 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.237+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2bb9c00 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea400 con 0x5645b2bb9c00 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea800 con 0x5645b2bb9c00 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.238+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feac00 con 0x5645b2f61000 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 zone default found 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 4 Realm: () 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:07.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 zonegroup default 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb000 con 0x5645b2f61400 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb400 con 0x5645b2f61000 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea400 con 0x5645b2f61000 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 started sync module instance, tier type = 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.239+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea800 con 0x5645b2fea000 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.240+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.240+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94857354377984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb000 con 0x5645b2fea000 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.240+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2febc00 con 0x5645b2bb9c00 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.241+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94857354377984] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.242+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.242+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94857354379136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61c00 con 0x5645b2bb9c00 2024-04-27T04:35:07.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.242+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61800 con 0x5645b2f61000 2024-04-27T04:35:07.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.243+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94857354379136] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.243+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.243+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94857354380288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2f61000 2024-04-27T04:35:07.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.243+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea800 con 0x5645b2fea000 2024-04-27T04:35:07.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.244+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94857354380288] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.244+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.244+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94857354381440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb000 con 0x5645b2fea000 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.244+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feac00 con 0x5645b2bb9c00 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.245+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.245+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94857354382592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb400 con 0x5645b2bb9c00 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.245+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea400 con 0x5645b2f61400 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.245+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94857354381440] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.246+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.246+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94857354382592] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.246+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94857354383744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f60400 con 0x5645b2f61400 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.246+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2000 con 0x5645b2f61400 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.247+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(56 notify.6 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.247+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94857354384896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2400 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.247+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2800 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(58 notify.7 [create] v33'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94857354383744] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94857354386048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2c00 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=0 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=1 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=2 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=3 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=4 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=5 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94857354384896] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.248+0000 7fc541eb2a80 20 add_watcher() i=6 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.249+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94857354386048] v33'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.249+0000 7fc541eb2a80 20 add_watcher() i=7 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.249+0000 7fc541eb2a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.249+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3000 con 0x5645b2bb9c00 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.249+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.249+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94857355829248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3400 con 0x5645b2bb9c00 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94857355829248] v33'197 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc526e2a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc541eb2a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc541eb2a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc541eb2a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.250+0000 7fc541eb2a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2c5a3c0 con 0x5645b2ff3c00 2024-04-27T04:35:07.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc53f65b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc53f65b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54742/0 (socket says 172.21.15.191:54742) 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc53f65b700 1 -- 172.21.15.191:0/3668629829 learned_addr learned my addr 172.21.15.191:0/3668629829 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc53f65b700 1 -- 172.21.15.191:0/3668629829 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be0b60 con 0x5645b2ff3c00 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc53f65b700 1 --2- 172.21.15.191:0/3668629829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5645b2f7d5f0 tx=0x5645b2f07f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f027b7a33e774697 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc526629700 1 -- 172.21.15.191:0/3668629829 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2ff8160 con 0x5645b2ff3c00 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc526629700 1 -- 172.21.15.191:0/3668629829 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b3014000 con 0x5645b2ff3c00 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.251+0000 7fc526629700 1 -- 172.21.15.191:0/3668629829 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2ff82c0 con 0x5645b2ff3c00 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 -- 172.21.15.191:0/3668629829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 msgr2=0x5645b2ea2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/3668629829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5645b2f7d5f0 tx=0x5645b2f07f80 comp rx=0 tx=0).stop 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 -- 172.21.15.191:0/3668629829 shutdown_connections 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/3668629829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 -- 172.21.15.191:0/3668629829 >> 172.21.15.191:0/3668629829 conn(0x5645b2ff3800 msgr2=0x5645b2ca6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 -- 172.21.15.191:0/3668629829 shutdown_connections 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 -- 172.21.15.191:0/3668629829 wait complete. 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 Processor -- start 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.252+0000 7fc541eb2a80 1 -- start start 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc541eb2a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc541eb2a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2c5a500 con 0x5645b2ff3c00 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc53f65b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc53f65b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:54748/0 (socket says 172.21.15.191:54748) 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 learned_addr learned my addr 172.21.15.191:0/4152016548 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2be0d00 con 0x5645b2ff3c00 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5645b300c210 tx=0x5645b2fde240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc524e26700 1 -- 172.21.15.191:0/4152016548 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2ff8420 con 0x5645b2ff3c00 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc524e26700 1 -- 172.21.15.191:0/4152016548 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645b3014180 con 0x5645b2ff3c00 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc524e26700 1 -- 172.21.15.191:0/4152016548 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645b2ff8160 con 0x5645b2ff3c00 2024-04-27T04:35:07.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645b2be0340 con 0x5645b2ff3c00 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.253+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b2be0b60 con 0x5645b2ff3c00 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc524e26700 1 -- 172.21.15.191:0/4152016548 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5645b395bb80 con 0x5645b2ff3c00 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc524e26700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2febc00 0x5645b2ea3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc524e26700 1 -- 172.21.15.191:0/4152016548 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5645b2c336c0 con 0x5645b2ff3c00 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc54065d700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2febc00 0x5645b2ea3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2f61c00 0x5645b30a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f60400 con 0x5645b2f61c00 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.254+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2f61c00 0x5645b30a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.255+0000 7fc54065d700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2febc00 0x5645b2ea3700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5645b2f7daa0 tx=0x5645b2fdf2c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.255+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2f61c00 0x5645b30a0000 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.256+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'58 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61c00 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.256+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2feac00 0x5645b30a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.256+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb400 con 0x5645b2feac00 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.256+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2feac00 0x5645b30a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.256+0000 7fc53f65b700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2feac00 0x5645b30a0580 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.257+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.257+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2fea400 0x5645b30a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.257+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002000 con 0x5645b2fea400 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.257+0000 7fc54065d700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2fea400 0x5645b30a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.257+0000 7fc54065d700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2fea400 0x5645b30a0b00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.258+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.258+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002400 con 0x5645b2f61c00 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.259+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'198 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61c00 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.259+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b3002800 0x5645b30a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.259+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002c00 con 0x5645b3002800 2024-04-27T04:35:07.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.260+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b3002800 0x5645b30a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.260+0000 7fc53fe5c700 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b3002800 0x5645b30a1080 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.261+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'74 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.261+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003000 con 0x5645b2fea400 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.261+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'56 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.262+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003400 con 0x5645b2f61c00 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.262+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61c00 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.262+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003800 con 0x5645b2feac00 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.263+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'51 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.263+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003c00 con 0x5645b3002800 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.264+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.264+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002400 con 0x5645b2fea400 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.265+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.265+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002c00 con 0x5645b3002800 2024-04-27T04:35:07.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.267+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'75 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.267+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002000 con 0x5645b2feac00 2024-04-27T04:35:07.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.269+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'30 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.269+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2000 con 0x5645b3002800 2024-04-27T04:35:07.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.270+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'37 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.270+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2400 con 0x5645b2feac00 2024-04-27T04:35:07.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.271+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'66 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.271+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2800 con 0x5645b2f61c00 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.272+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'199 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61c00 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.272+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2c00 con 0x5645b2fea400 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.273+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'65 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.273+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3000 con 0x5645b3002800 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.274+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'76 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.274+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3400 con 0x5645b2fea400 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.275+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'37 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.275+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f61800 con 0x5645b2f61c00 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.276+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61c00 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.276+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2f60400 con 0x5645b2f61c00 2024-04-27T04:35:07.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.276+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'200 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61c00 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.277+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.277+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'66 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.277+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003400 con 0x5645b3002800 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.278+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'41 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.278+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003c00 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.279+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.279+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002c00 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.280+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'43 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.280+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003800 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.281+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.281+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002000 con 0x5645b2fea400 2024-04-27T04:35:07.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.282+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'57 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.282+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea800 con 0x5645b2feac00 2024-04-27T04:35:07.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.283+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.283+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb400 con 0x5645b2fea400 2024-04-27T04:35:07.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.284+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'37 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.284+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2000 con 0x5645b2feac00 2024-04-27T04:35:07.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.285+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'52 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.285+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2800 con 0x5645b2fea400 2024-04-27T04:35:07.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.286+0000 7fc54065d700 1 -- 172.21.15.191:0/4152016548 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'67 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2fea400 2024-04-27T04:35:07.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.286+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3000 con 0x5645b2feac00 2024-04-27T04:35:07.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.287+0000 7fc53f65b700 1 -- 172.21.15.191:0/4152016548 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'67 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2feac00 2024-04-27T04:35:07.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.287+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003000 con 0x5645b3002800 2024-04-27T04:35:07.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.288+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b3002800 2024-04-27T04:35:07.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc522621700 20 reqs_thread_entry: start 2024-04-27T04:35:07.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc541eb2a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:07.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc541eb2a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc52061d700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:35:37 2024 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc51fdfb700 20 reqs_thread_entry: start 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc541eb2a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc541eb2a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.289+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003c00 con 0x5645b2bb9c00 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc541eb2a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc541eb2a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc541eb2a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc541eb2a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002c00 con 0x5645b2fea000 2024-04-27T04:35:07.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.290+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv42 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.293 INFO:teuthology.orchestra.run.smithi191.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T04:35:07.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.291+0000 7fc526e2a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:07.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.291+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94857355829248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3003800 con 0x5645b2bb9c00 2024-04-27T04:35:07.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.292+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94857355829248] v33'201 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.292+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94857354377984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2feb800 con 0x5645b2fea000 2024-04-27T04:35:07.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.293+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94857354377984] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.293+0000 7fc541eb2a80 20 remove_watcher() i=0 2024-04-27T04:35:07.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.293+0000 7fc541eb2a80 2 removed watcher, disabling cache 2024-04-27T04:35:07.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.293+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94857354379136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2c00 con 0x5645b2bb9c00 2024-04-27T04:35:07.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.294+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94857354379136] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.294+0000 7fc541eb2a80 20 remove_watcher() i=1 2024-04-27T04:35:07.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.294+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94857354380288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3400 con 0x5645b2f61000 2024-04-27T04:35:07.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.295+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4047299336 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94857354380288] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7200 con 0x5645b2f61000 2024-04-27T04:35:07.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.295+0000 7fc541eb2a80 20 remove_watcher() i=2 2024-04-27T04:35:07.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.295+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94857354381440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b3002c00 con 0x5645b2fea000 2024-04-27T04:35:07.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.296+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94857354381440] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd7440 con 0x5645b2fea000 2024-04-27T04:35:07.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.296+0000 7fc541eb2a80 20 remove_watcher() i=3 2024-04-27T04:35:07.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.296+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94857354382592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2400 con 0x5645b2bb9c00 2024-04-27T04:35:07.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.297+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94857354382592] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2bb9c00 2024-04-27T04:35:07.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.297+0000 7fc541eb2a80 20 remove_watcher() i=4 2024-04-27T04:35:07.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.297+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94857354383744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff2000 con 0x5645b2f61400 2024-04-27T04:35:07.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.298+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94857354383744] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.298+0000 7fc541eb2a80 20 remove_watcher() i=5 2024-04-27T04:35:07.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.298+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94857354384896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2ff3000 con 0x5645b2f61400 2024-04-27T04:35:07.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.299+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94857354384896] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.299+0000 7fc541eb2a80 20 remove_watcher() i=6 2024-04-27T04:35:07.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.299+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94857354386048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5645b2fea800 con 0x5645b2f61400 2024-04-27T04:35:07.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.299+0000 7fc54065d700 1 -- 172.21.15.191:0/4047299336 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94857354386048] v33'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645b2fd6b40 con 0x5645b2f61400 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.299+0000 7fc541eb2a80 20 remove_watcher() i=7 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2f61400 msgr2=0x5645b2ea3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2f61400 0x5645b2ea3180 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2bb9c00 msgr2=0x5645b2ea1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2bb9c00 0x5645b2ea1b80 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b2fea000 msgr2=0x5645b2ea2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b2fea000 0x5645b2ea2c00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2f61000 msgr2=0x5645b2ea2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2f61000 0x5645b2ea2100 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2f60c00 msgr2=0x5645b2ea1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2f60c00 0x5645b2ea1600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5645b3a304e0 tx=0x5645b2fde000 comp rx=0 tx=0).stop 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc53f65b700 1 -- 172.21.15.191:0/4047299336 reap_dead start 2024-04-27T04:35:07.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 msgr2=0x5645b2ea0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5645b2f40600 tx=0x5645b2f06e40 comp rx=0 tx=0).stop 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 shutdown_connections 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4047299336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb9800 0x5645b2ea0b00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 >> 172.21.15.191:0/4047299336 conn(0x5645b2bb8c00 msgr2=0x5645b2ca5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 shutdown_connections 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4047299336 wait complete. 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2feac00 msgr2=0x5645b30a0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5645b2feac00 0x5645b30a0580 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2f61c00 msgr2=0x5645b30a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5645b2f61c00 0x5645b30a0000 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b3002800 msgr2=0x5645b30a1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5645b3002800 0x5645b30a1080 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2fea400 msgr2=0x5645b30a0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5645b2fea400 0x5645b30a0b00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2febc00 msgr2=0x5645b2ea3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2febc00 0x5645b2ea3700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5645b2f7daa0 tx=0x5645b2fdf2c0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 msgr2=0x5645b2ea2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc53fe5c700 1 -- 172.21.15.191:0/4152016548 reap_dead start 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5645b300c210 tx=0x5645b2fde240 comp rx=0 tx=0).stop 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 shutdown_connections 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/4152016548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2ff3c00 0x5645b2ea2680 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 >> 172.21.15.191:0/4152016548 conn(0x5645b2ff3800 msgr2=0x5645b2ca6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.300+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 shutdown_connections 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/4152016548 wait complete. 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2bb9400 msgr2=0x5645b2ea1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2bb9400 0x5645b2ea1080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5645b2f40030 tx=0x5645b2f06b40 comp rx=0 tx=0).stop 2024-04-27T04:35:07.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 msgr2=0x5645b2ea0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5645b3a23ec0 tx=0x5645b2ca9920 comp rx=0 tx=0).stop 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 shutdown_connections 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5645b2bb9400 0x5645b2ea1080 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 --2- 172.21.15.191:0/1699907241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b2bb8800 0x5645b2ea0580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 >> 172.21.15.191:0/1699907241 conn(0x5645b2bb8400 msgr2=0x5645b2ca5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 shutdown_connections 2024-04-27T04:35:07.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:07.301+0000 7fc541eb2a80 1 -- 172.21.15.191:0/1699907241 wait complete. 2024-04-27T04:35:07.308 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T04:35:07.311 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:07.315 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-27T04:35:42.316 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-04-27T04:35:42.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-04-27T04:35:42.317 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-04-27T04:35:42.371 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:42.371 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4d981a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x558ba5684c00 0x558ba596c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4d981a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558ba59af500 con 0x558ba5684800 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4d981a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x558ba5726a00 con 0x558ba5684c00 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4b12a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x558ba5684c00 0x558ba596c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4b92b700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x558ba5684800 0x558ba5770800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52624/0 (socket says 172.21.15.191:52624) 2024-04-27T04:35:42.372 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.370+0000 7efd4b92b700 1 -- 172.21.15.191:0/4441894 learned_addr learned my addr 172.21.15.191:0/4441894 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.371+0000 7efd4b12a700 1 -- 172.21.15.191:0/4441894 >> v1:172.21.15.138:6789/0 conn(0x558ba5684800 legacy=0x558ba5770800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.371+0000 7efd4b12a700 1 -- 172.21.15.191:0/4441894 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56add40 con 0x558ba5684c00 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.371+0000 7efd4b12a700 1 --2- 172.21.15.191:0/4441894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684c00 0x558ba596c580 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x558ba5967890 tx=0x558ba5775920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4facfb270c86e699 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.371+0000 7efd4a929700 1 -- 172.21.15.191:0/4441894 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d4a0 con 0x558ba5684c00 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.371+0000 7efd4a929700 1 -- 172.21.15.191:0/4441894 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba59af800 con 0x558ba5684c00 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.371+0000 7efd4a929700 1 -- 172.21.15.191:0/4441894 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d600 con 0x558ba5684c00 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 -- 172.21.15.191:0/4441894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684c00 msgr2=0x558ba596c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 --2- 172.21.15.191:0/4441894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684c00 0x558ba596c580 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x558ba5967890 tx=0x558ba5775920 comp rx=0 tx=0).stop 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 -- 172.21.15.191:0/4441894 shutdown_connections 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 --2- 172.21.15.191:0/4441894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684c00 0x558ba596c580 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.375 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 -- 172.21.15.191:0/4441894 >> 172.21.15.191:0/4441894 conn(0x558ba5684400 msgr2=0x558ba5770000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 -- 172.21.15.191:0/4441894 shutdown_connections 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 -- 172.21.15.191:0/4441894 wait complete. 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.372+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4d981a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4d981a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba5726500 con 0x558ba5684800 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4b92b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4b92b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58336/0 (socket says 172.21.15.191:58336) 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4b92b700 1 -- 172.21.15.191:0/1984612039 learned_addr learned my addr 172.21.15.191:0/1984612039 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4b92b700 1 -- 172.21.15.191:0/1984612039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56adba0 con 0x558ba5684800 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4b92b700 1 --2- 172.21.15.191:0/1984612039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x558ba645d890 tx=0x558ba5775b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc00a54f2ceeafa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4a128700 1 -- 172.21.15.191:0/1984612039 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d4a0 con 0x558ba5684800 2024-04-27T04:35:42.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4a128700 1 -- 172.21.15.191:0/1984612039 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba59afb00 con 0x558ba5684800 2024-04-27T04:35:42.377 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.373+0000 7efd4a128700 1 -- 172.21.15.191:0/1984612039 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d600 con 0x558ba5684800 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- 172.21.15.191:0/1984612039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 msgr2=0x558ba596c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1984612039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x558ba645d890 tx=0x558ba5775b60 comp rx=0 tx=0).stop 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- 172.21.15.191:0/1984612039 shutdown_connections 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1984612039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- 172.21.15.191:0/1984612039 >> 172.21.15.191:0/1984612039 conn(0x558ba5684400 msgr2=0x558ba5771000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- 172.21.15.191:0/1984612039 shutdown_connections 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- 172.21.15.191:0/1984612039 wait complete. 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4d981a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba57263c0 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.374+0000 7efd4b92b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd4b92b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58340/0 (socket says 172.21.15.191:58340) 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd4b92b700 1 -- 172.21.15.191:0/1394846917 learned_addr learned my addr 172.21.15.191:0/1394846917 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd4b92b700 1 -- 172.21.15.191:0/1394846917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56ada00 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd4b92b700 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x558ba645da70 tx=0x558ba5775a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd48925700 1 -- 172.21.15.191:0/1394846917 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d8c0 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd48925700 1 -- 172.21.15.191:0/1394846917 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba5725c80 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558ba56ad860 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd48925700 1 -- 172.21.15.191:0/1394846917 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d4a0 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.375+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ba56ad6c0 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd48925700 1 -- 172.21.15.191:0/1394846917 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558ba6426b00 con 0x558ba5684800 2024-04-27T04:35:42.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4d981a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4d981a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba5726000 con 0x558ba5685400 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd48925700 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14000 0x558ba596d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd48925700 1 -- 172.21.15.191:0/1394846917 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x558ba56ff180 con 0x558ba5684800 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4b92b700 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14000 0x558ba596d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4c12c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4c12c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58354/0 (socket says 172.21.15.191:58354) 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.376+0000 7efd4c12c700 1 -- 172.21.15.191:0/1714666777 learned_addr learned my addr 172.21.15.191:0/1714666777 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4b92b700 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14000 0x558ba596d080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x558ba6502bd0 tx=0x558ba59d0240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4c12c700 1 -- 172.21.15.191:0/1714666777 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56ac000 con 0x558ba5685400 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4c12c700 1 --2- 172.21.15.191:0/1714666777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x558ba6502660 tx=0x558ba59d10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4d0648ac7c936e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd46120700 1 -- 172.21.15.191:0/1714666777 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d8c0 con 0x558ba5685400 2024-04-27T04:35:42.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd46120700 1 -- 172.21.15.191:0/1714666777 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba59ae300 con 0x558ba5685400 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd46120700 1 -- 172.21.15.191:0/1714666777 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d4a0 con 0x558ba5685400 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 -- 172.21.15.191:0/1714666777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 msgr2=0x558ba596cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1714666777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x558ba6502660 tx=0x558ba59d10e0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 -- 172.21.15.191:0/1714666777 shutdown_connections 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1714666777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 -- 172.21.15.191:0/1714666777 >> 172.21.15.191:0/1714666777 conn(0x558ba5684c00 msgr2=0x558ba5771800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 -- 172.21.15.191:0/1714666777 shutdown_connections 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.377+0000 7efd4d981a80 1 -- 172.21.15.191:0/1714666777 wait complete. 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4d981a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4d981a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba56c0640 con 0x558ba5685400 2024-04-27T04:35:42.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4c12c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4c12c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58362/0 (socket says 172.21.15.191:58362) 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 learned_addr learned my addr 172.21.15.191:0/3681168328 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.378+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56acea0 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x558ba6503f80 tx=0x558ba59d0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4491d700 1 -- 172.21.15.191:0/3681168328 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573db80 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4491d700 1 -- 172.21.15.191:0/3681168328 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba5725c80 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558ba56ac820 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4491d700 1 -- 172.21.15.191:0/3681168328 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d8c0 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ba56ac9c0 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.379+0000 7efd4491d700 1 -- 172.21.15.191:0/3681168328 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558ba6426dc0 con 0x558ba5685400 2024-04-27T04:35:42.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4491d700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14c00 0x558ba596d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4491d700 1 -- 172.21.15.191:0/3681168328 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x558ba56ff340 con 0x558ba5685400 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4b12a700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14c00 0x558ba596d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4b12a700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14c00 0x558ba596d600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x558ba5967890 tx=0x558ba5775800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558ba5685800 0x558ba596db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5685c00 con 0x558ba5685800 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.380+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558ba5685800 0x558ba596db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558ba5685800 0x558ba596db80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 20 realm 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15000 con 0x558ba5685800 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15400 con 0x558ba5685800 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.381+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15800 con 0x558ba5685800 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4d981a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558ba5a15c00 0x558ba596e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15000 con 0x558ba5a15c00 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4b92b700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558ba5a15c00 0x558ba596e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.382+0000 7efd4b92b700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558ba5a15c00 0x558ba596e100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4d981a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4d981a80 20 searching for the correct realm 2024-04-27T04:35:42.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15400 con 0x558ba5a15c00 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4b92b700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558ba5a15400 0x558ba596ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15000 con 0x558ba5a15400 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4b12a700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558ba5a15400 0x558ba596ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.383+0000 7efd4b12a700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558ba5a15400 0x558ba596ec00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.384+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.384+0000 7efd4b12a700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558ba5a15000 0x558ba596f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.384+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.384+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558ba5a15000 0x558ba596f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.384+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558ba5a15000 0x558ba596f180 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.385+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15400 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.386+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15400 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.387+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.388+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.389+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15400 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.390+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15400 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15400 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15c00 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.391+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15400 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:42.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5685c00 con 0x558ba5685800 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.392+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc400 con 0x558ba5685800 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc800 con 0x558ba5a15c00 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 zone default found 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 4 Realm: () 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:42.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 zonegroup default 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adcc00 con 0x558ba5a15000 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add000 con 0x558ba5a15c00 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.393+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add400 con 0x558ba5a15c00 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.394+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.394+0000 7efd4d981a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.394+0000 7efd4d981a80 20 started sync module instance, tier type = 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.394+0000 7efd4d981a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.394+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add800 con 0x558ba5a15400 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.395+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.395+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94058266901248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc800 con 0x558ba5a15400 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.395+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add000 con 0x558ba5685800 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.396+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94058266901248] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.396+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.396+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94058266902400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add400 con 0x558ba5685800 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.396+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc400 con 0x558ba5a15c00 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.398+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94058266902400] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.398+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.398+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94058266903552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adcc00 con 0x558ba5a15c00 2024-04-27T04:35:42.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.398+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5685c00 con 0x558ba5a15400 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.399+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.399+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94058266904704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5a15800 con 0x558ba5a15400 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.399+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94058266903552] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.399+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5685800 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.400+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94058266904704] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.400+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.400+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94058266905856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add800 con 0x558ba5685800 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.400+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc800 con 0x558ba5a15000 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.401+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94058266905856] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.401+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.402+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94058266907008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5addc00 con 0x558ba5a15000 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.402+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add000 con 0x558ba5a15000 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.403+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94058266907008] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.403+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.403+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94058266908160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add400 con 0x558ba5a15000 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.403+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc000 con 0x558ba5a15000 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94058266908160] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94058266909312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add800 con 0x558ba5a15000 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=0 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=1 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=2 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=3 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=4 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=5 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.404+0000 7efd4d981a80 20 add_watcher() i=6 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.405+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94058266909312] v33'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.405+0000 7efd4d981a80 20 add_watcher() i=7 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.405+0000 7efd4d981a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.405+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc800 con 0x558ba5685800 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.405+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.405+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94058268499968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5addc00 con 0x558ba5685800 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.406+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94058268499968] v33'202 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.406+0000 7efd328f9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.406+0000 7efd4d981a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.406+0000 7efd4d981a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4d981a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4d981a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba56c0780 con 0x558ba5add400 2024-04-27T04:35:42.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4b12a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4b12a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58368/0 (socket says 172.21.15.191:58368) 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4b12a700 1 -- 172.21.15.191:0/2621934107 learned_addr learned my addr 172.21.15.191:0/2621934107 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4b12a700 1 -- 172.21.15.191:0/2621934107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56acb60 con 0x558ba5add400 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.407+0000 7efd4b12a700 1 --2- 172.21.15.191:0/2621934107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x558ba5a7b6e0 tx=0x558ba5775980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a396e1cd7f3fe1e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd320f8700 1 -- 172.21.15.191:0/2621934107 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba573d760 con 0x558ba5add400 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd320f8700 1 -- 172.21.15.191:0/2621934107 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba5af8000 con 0x558ba5add400 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd320f8700 1 -- 172.21.15.191:0/2621934107 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba5afa000 con 0x558ba5add400 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 -- 172.21.15.191:0/2621934107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 msgr2=0x558ba596e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 --2- 172.21.15.191:0/2621934107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x558ba5a7b6e0 tx=0x558ba5775980 comp rx=0 tx=0).stop 2024-04-27T04:35:42.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 -- 172.21.15.191:0/2621934107 shutdown_connections 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 --2- 172.21.15.191:0/2621934107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 -- 172.21.15.191:0/2621934107 >> 172.21.15.191:0/2621934107 conn(0x558ba5add000 msgr2=0x558ba5772000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 -- 172.21.15.191:0/2621934107 shutdown_connections 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.408+0000 7efd4d981a80 1 -- 172.21.15.191:0/2621934107 wait complete. 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4d981a80 1 Processor -- start 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4d981a80 1 -- start start 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4d981a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4d981a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba5726000 con 0x558ba5add400 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4b12a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4b12a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58382/0 (socket says 172.21.15.191:58382) 2024-04-27T04:35:42.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4b12a700 1 -- 172.21.15.191:0/3064600382 learned_addr learned my addr 172.21.15.191:0/3064600382 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.409+0000 7efd4b12a700 1 -- 172.21.15.191:0/3064600382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba56acd00 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd4b12a700 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x558ba5af2210 tx=0x558ba5abc8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd308f5700 1 -- 172.21.15.191:0/3064600382 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba5afa160 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd308f5700 1 -- 172.21.15.191:0/3064600382 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ba5af8180 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558ba56ac340 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd308f5700 1 -- 172.21.15.191:0/3064600382 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558ba5afa2c0 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ba56add40 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.410+0000 7efd308f5700 1 -- 172.21.15.191:0/3064600382 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558ba6427b80 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.411+0000 7efd308f5700 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5b76000 0x558ba596f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.411+0000 7efd308f5700 1 -- 172.21.15.191:0/3064600382 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x558ba56ff500 con 0x558ba5add400 2024-04-27T04:35:42.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.411+0000 7efd4d981a80 5 note: GC not initialized 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.411+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5b76000 0x558ba596f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.411+0000 7efd4c12c700 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5b76000 0x558ba596f700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x558ba5a7a180 tx=0x558ba5abc5a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd2e0f0700 20 reqs_thread_entry: start 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd4d981a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd4d981a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd2c0ec700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:12 2024 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd2b8ca700 20 reqs_thread_entry: start 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add800 con 0x558ba5685800 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.412+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc800 con 0x558ba5a15400 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76400 con 0x558ba5685800 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76800 con 0x558ba5a15c00 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76c00 con 0x558ba5a15c00 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.413+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77000 con 0x558ba5a15c00 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77400 con 0x558ba5685800 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77800 con 0x558ba5685800 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77c00 con 0x558ba5a15c00 2024-04-27T04:35:42.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.414+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76800 con 0x558ba5a15c00 2024-04-27T04:35:42.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.415+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.415+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76c00 con 0x558ba5a15000 2024-04-27T04:35:42.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.415+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.415+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc400 con 0x558ba5685800 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.415+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adcc00 con 0x558ba5a15400 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5addc00 con 0x558ba5a15c00 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add800 con 0x558ba5a15000 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.416+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc400 con 0x558ba5a15000 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.417+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.417+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76400 con 0x558ba5a15c00 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.417+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.417+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77400 con 0x558ba5a15000 2024-04-27T04:35:42.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.417+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.417+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77800 con 0x558ba5685800 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76c00 con 0x558ba5a15c00 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77000 con 0x558ba5a15000 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77c00 con 0x558ba5a15400 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.418+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76800 con 0x558ba5a15000 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76400 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5addc00 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.419+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77400 con 0x558ba5a15c00 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77800 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77000 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76800 con 0x558ba5a15c00 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76400 con 0x558ba5a15c00 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.420+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77c00 con 0x558ba5a15c00 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.421+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.421+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5add800 con 0x558ba5a15000 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.421+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.422+0000 7efd328f9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.422+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94058268499968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc400 con 0x558ba5685800 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.423+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94058268499968] v33'203 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.423+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94058266901248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5addc00 con 0x558ba5a15400 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.424+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94058266901248] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.424+0000 7efd4d981a80 20 remove_watcher() i=0 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.424+0000 7efd4d981a80 2 removed watcher, disabling cache 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.424+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94058266902400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adc800 con 0x558ba5685800 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.425+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94058266902400] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.425+0000 7efd4d981a80 20 remove_watcher() i=1 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.425+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94058266903552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adcc00 con 0x558ba5a15c00 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.426+0000 7efd4b92b700 1 -- 172.21.15.191:0/3681168328 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94058266903552] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1200 con 0x558ba5a15c00 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.426+0000 7efd4d981a80 20 remove_watcher() i=2 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.426+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94058266904704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5685c00 con 0x558ba5a15400 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.427+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94058266904704] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad1440 con 0x558ba5a15400 2024-04-27T04:35:42.455 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.427+0000 7efd4d981a80 20 remove_watcher() i=3 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.427+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94058266905856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5adcc00 con 0x558ba5685800 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.428+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94058266905856] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5685800 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.428+0000 7efd4d981a80 20 remove_watcher() i=4 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.428+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94058266907008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76c00 con 0x558ba5a15000 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.429+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94058266907008] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.429+0000 7efd4d981a80 20 remove_watcher() i=5 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.430+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94058266908160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b77400 con 0x558ba5a15000 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.431+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94058266908160] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.431+0000 7efd4d981a80 20 remove_watcher() i=6 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.431+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94058266909312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558ba5b76800 con 0x558ba5a15000 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4c12c700 1 -- 172.21.15.191:0/3681168328 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94058266909312] v33'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558ba5ad0b40 con 0x558ba5a15000 2024-04-27T04:35:42.456 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 20 remove_watcher() i=7 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558ba5a15000 msgr2=0x558ba596f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x558ba5a15000 0x558ba596f180 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558ba5685800 msgr2=0x558ba596db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x558ba5685800 0x558ba596db80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558ba5a15400 msgr2=0x558ba596ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x558ba5a15400 0x558ba596ec00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558ba5a15c00 msgr2=0x558ba596e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x558ba5a15c00 0x558ba596e100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14c00 msgr2=0x558ba596d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14c00 0x558ba596d600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x558ba5967890 tx=0x558ba5775800 comp rx=0 tx=0).stop 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 msgr2=0x558ba596cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3681168328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5685400 0x558ba596cb00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x558ba6503f80 tx=0x558ba59d0cc0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 reap_dead start 2024-04-27T04:35:42.457 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4b12a700 1 -- 172.21.15.191:0/3681168328 reap_dead start 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 shutdown_connections 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 >> 172.21.15.191:0/3681168328 conn(0x558ba5684c00 msgr2=0x558ba5771800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 shutdown_connections 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.432+0000 7efd4d981a80 1 -- 172.21.15.191:0/3681168328 wait complete. 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5b76000 msgr2=0x558ba596f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5b76000 0x558ba596f700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x558ba5a7a180 tx=0x558ba5abc5a0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 msgr2=0x558ba596e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x558ba5af2210 tx=0x558ba5abc8a0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 shutdown_connections 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5b76000 0x558ba596f700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/3064600382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5add400 0x558ba596e680 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 >> 172.21.15.191:0/3064600382 conn(0x558ba5add000 msgr2=0x558ba5772000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 shutdown_connections 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/3064600382 wait complete. 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14000 msgr2=0x558ba596d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14000 0x558ba596d080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x558ba6502bd0 tx=0x558ba59d0240 comp rx=0 tx=0).stop 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 msgr2=0x558ba596c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x558ba645da70 tx=0x558ba5775a40 comp rx=0 tx=0).stop 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 shutdown_connections 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x558ba5a14000 0x558ba596d080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 --2- 172.21.15.191:0/1394846917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ba5684800 0x558ba596c580 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 >> 172.21.15.191:0/1394846917 conn(0x558ba5684400 msgr2=0x558ba5771000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 shutdown_connections 2024-04-27T04:35:42.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.433+0000 7efd4d981a80 1 -- 172.21.15.191:0/1394846917 wait complete. 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "anonymous", 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "anonymous", 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.461 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.462 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.463 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.464 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.465 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "anonymous", 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [], 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 0, 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 0 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.467 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.468 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.469 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.469 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.469 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.469 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:42.469 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'anonymous', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T04:35:42.470 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-27T04:35:42.470 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-27T04:35:42.470 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-27T04:35:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:42.514 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c655ea80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55e218988800 0x55e218c70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c655ea80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e218cb3500 con 0x55e218988c00 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c655ea80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e218a2aa00 con 0x55e218988800 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c3d07700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55e218988c00 0x55e218a74800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52626/0 (socket says 172.21.15.191:52626) 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c3d07700 1 -- 172.21.15.191:0/1487011601 learned_addr learned my addr 172.21.15.191:0/1487011601 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.513+0000 7f45c4508700 1 --2- 172.21.15.191:0/1487011601 >> v2:172.21.15.138:3300/0 conn(0x55e218988800 0x55e218c70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.514+0000 7f45c4508700 1 -- 172.21.15.191:0/1487011601 >> v1:172.21.15.138:6789/0 conn(0x55e218988c00 legacy=0x55e218a74800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.514+0000 7f45c4508700 1 -- 172.21.15.191:0/1487011601 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b1d40 con 0x55e218988800 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.514+0000 7f45c4508700 1 --2- 172.21.15.191:0/1487011601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55e218c6b9e0 tx=0x55e218a79920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e782a8bc4820b9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.514+0000 7f45c3506700 1 -- 172.21.15.191:0/1487011601 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a414a0 con 0x55e218988800 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.514+0000 7f45c3506700 1 -- 172.21.15.191:0/1487011601 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218cb3800 con 0x55e218988800 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.514+0000 7f45c3506700 1 -- 172.21.15.191:0/1487011601 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a41600 con 0x55e218988800 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- 172.21.15.191:0/1487011601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 msgr2=0x55e218c70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 --2- 172.21.15.191:0/1487011601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55e218c6b9e0 tx=0x55e218a79920 comp rx=0 tx=0).stop 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- 172.21.15.191:0/1487011601 shutdown_connections 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 --2- 172.21.15.191:0/1487011601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- 172.21.15.191:0/1487011601 >> 172.21.15.191:0/1487011601 conn(0x55e218988400 msgr2=0x55e218a74000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- 172.21.15.191:0/1487011601 shutdown_connections 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- 172.21.15.191:0/1487011601 wait complete. 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c655ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e218a2a500 con 0x55e218988800 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c4508700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.515+0000 7f45c4508700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58402/0 (socket says 172.21.15.191:58402) 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c4508700 1 -- 172.21.15.191:0/1223418543 learned_addr learned my addr 172.21.15.191:0/1223418543 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c4508700 1 -- 172.21.15.191:0/1223418543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b1ba0 con 0x55e218988800 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c4508700 1 --2- 172.21.15.191:0/1223418543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55e219753740 tx=0x55e218a79800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30298e5930067a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c2d05700 1 -- 172.21.15.191:0/1223418543 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a41760 con 0x55e218988800 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c2d05700 1 -- 172.21.15.191:0/1223418543 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218cb3b00 con 0x55e218988800 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c2d05700 1 -- 172.21.15.191:0/1223418543 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a411e0 con 0x55e218988800 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 -- 172.21.15.191:0/1223418543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 msgr2=0x55e218c70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 --2- 172.21.15.191:0/1223418543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55e219753740 tx=0x55e218a79800 comp rx=0 tx=0).stop 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 -- 172.21.15.191:0/1223418543 shutdown_connections 2024-04-27T04:35:42.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 --2- 172.21.15.191:0/1223418543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 -- 172.21.15.191:0/1223418543 >> 172.21.15.191:0/1223418543 conn(0x55e218988400 msgr2=0x55e218a75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 -- 172.21.15.191:0/1223418543 shutdown_connections 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.516+0000 7f45c655ea80 1 -- 172.21.15.191:0/1223418543 wait complete. 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c655ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c655ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e218a2a3c0 con 0x55e218988800 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c4508700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c4508700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58406/0 (socket says 172.21.15.191:58406) 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c4508700 1 -- 172.21.15.191:0/3880159122 learned_addr learned my addr 172.21.15.191:0/3880159122 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c4508700 1 -- 172.21.15.191:0/3880159122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b1a00 con 0x55e218988800 2024-04-27T04:35:42.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.517+0000 7f45c4508700 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55e2197535f0 tx=0x55e218a79920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c1502700 1 -- 172.21.15.191:0/3880159122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a414a0 con 0x55e218988800 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c1502700 1 -- 172.21.15.191:0/3880159122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218cb3c80 con 0x55e218988800 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2189b1860 con 0x55e218988800 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c1502700 1 -- 172.21.15.191:0/3880159122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a41600 con 0x55e218988800 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2189b16c0 con 0x55e218988800 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c655ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.518+0000 7f45c655ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e218a2a000 con 0x55e218989400 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c1502700 1 -- 172.21.15.191:0/3880159122 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e21972ab00 con 0x55e218988800 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4d09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4d09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58414/0 (socket says 172.21.15.191:58414) 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4d09700 1 -- 172.21.15.191:0/1126110263 learned_addr learned my addr 172.21.15.191:0/1126110263 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c1502700 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18000 0x55e218c71080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c1502700 1 -- 172.21.15.191:0/3880159122 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e218a03180 con 0x55e218988800 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4508700 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18000 0x55e218c71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4d09700 1 -- 172.21.15.191:0/1126110263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b0000 con 0x55e218989400 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4d09700 1 --2- 172.21.15.191:0/1126110263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55e21981fc80 tx=0x55e218ce0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec4a8c10737416e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45c4508700 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18000 0x55e218c71080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e21981ef60 tx=0x55e218ce0c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45becfd700 1 -- 172.21.15.191:0/1126110263 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a414a0 con 0x55e218989400 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45becfd700 1 -- 172.21.15.191:0/1126110263 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218a29c80 con 0x55e218989400 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.519+0000 7f45becfd700 1 -- 172.21.15.191:0/1126110263 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a41600 con 0x55e218989400 2024-04-27T04:35:42.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- 172.21.15.191:0/1126110263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 msgr2=0x55e218c70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 --2- 172.21.15.191:0/1126110263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55e21981fc80 tx=0x55e218ce0b40 comp rx=0 tx=0).stop 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- 172.21.15.191:0/1126110263 shutdown_connections 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 --2- 172.21.15.191:0/1126110263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- 172.21.15.191:0/1126110263 >> 172.21.15.191:0/1126110263 conn(0x55e218988c00 msgr2=0x55e218a75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- 172.21.15.191:0/1126110263 shutdown_connections 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- 172.21.15.191:0/1126110263 wait complete. 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c655ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2189c4640 con 0x55e218989400 2024-04-27T04:35:42.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c4d09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c4d09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58418/0 (socket says 172.21.15.191:58418) 2024-04-27T04:35:42.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 learned_addr learned my addr 172.21.15.191:0/2793448244 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.520+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b0ea0 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55e21981e0f0 tx=0x55e218ce05a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45bd4fa700 1 -- 172.21.15.191:0/2793448244 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a41760 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45bd4fa700 1 -- 172.21.15.191:0/2793448244 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218a29b00 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2189b0820 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45bd4fa700 1 -- 172.21.15.191:0/2793448244 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218a414a0 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2189b09c0 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.521+0000 7f45bd4fa700 1 -- 172.21.15.191:0/2793448244 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e21972adc0 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45bd4fa700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18c00 0x55e218c71600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45bd4fa700 1 -- 172.21.15.191:0/2793448244 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e218a03340 con 0x55e218989400 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18c00 0x55e218c71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e218989800 0x55e218c71b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218989c00 con 0x55e218989800 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e218989800 0x55e218c71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18c00 0x55e218c71600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e218d1b770 tx=0x55e218dbe0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.522+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e218989800 0x55e218c71b80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 realm 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19000 con 0x55e218989800 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19400 con 0x55e218989800 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19800 con 0x55e218989800 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e218d19c00 0x55e218c72100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19000 con 0x55e218d19c00 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.523+0000 7f45c4508700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e218d19c00 0x55e218c72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c4508700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e218d19c00 0x55e218c72100 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c655ea80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c655ea80 20 searching for the correct realm 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19400 con 0x55e218d19c00 2024-04-27T04:35:42.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c4508700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e218d19400 0x55e218c72c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19000 con 0x55e218d19400 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.524+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e218d19400 0x55e218c72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.525+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e218d19400 0x55e218c72c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.525+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.525+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e218d19000 0x55e218c73180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.525+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.525+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e218d19000 0x55e218c73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.525+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e218d19000 0x55e218c73180 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.526+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19400 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.527+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19400 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.528+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.529+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19400 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.530+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19400 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19400 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.531+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19c00 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19400 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c655ea80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c655ea80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c655ea80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c655ea80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.532+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218989c00 con 0x55e218989800 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda400 con 0x55e218989800 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda800 con 0x55e218d19c00 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 zone default found 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 4 Realm: () 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 zonegroup default 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.533+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddac00 con 0x55e218d19000 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb000 con 0x55e218d19c00 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb400 con 0x55e218d19c00 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 started sync module instance, tier type = 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.534+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb800 con 0x55e218d19400 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.535+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.535+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94429566630656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda800 con 0x55e218d19400 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.535+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb000 con 0x55e218989800 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.536+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94429566630656] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.536+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.536+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94429566631808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb400 con 0x55e218989800 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.536+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda400 con 0x55e218d19c00 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.537+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.537+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94429566631808] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.537+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94429566632960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddac00 con 0x55e218d19c00 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.537+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218989c00 con 0x55e218d19400 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.538+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.539+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94429566634112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218d19800 con 0x55e218d19400 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.539+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218989800 2024-04-27T04:35:42.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.539+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94429566632960] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.540+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.540+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94429566635264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb800 con 0x55e218989800 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.540+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda800 con 0x55e218d19000 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.540+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94429566634112] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.540+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.541+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94429566636416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddbc00 con 0x55e218d19000 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.541+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb000 con 0x55e218d19000 2024-04-27T04:35:42.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.541+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94429566635264] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.541+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94429566636416] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.542+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.542+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94429566637568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb400 con 0x55e218d19000 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.542+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda000 con 0x55e218d19000 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(58 notify.7 [create] v33'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94429566638720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb800 con 0x55e218d19000 2024-04-27T04:35:42.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=0 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=1 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=2 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=3 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=4 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=5 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94429566637568] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.582 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.543+0000 7f45c655ea80 20 add_watcher() i=6 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.544+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94429566638720] v33'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.544+0000 7f45c655ea80 20 add_watcher() i=7 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.544+0000 7f45c655ea80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.544+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda800 con 0x55e218989800 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.544+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.544+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94429568196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddbc00 con 0x55e218989800 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94429568196608] v33'204 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.583 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45ab4d6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c655ea80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c655ea80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c655ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.545+0000 7f45c655ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2189c4780 con 0x55e218ddb400 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45c3d07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.584 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45c3d07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58428/0 (socket says 172.21.15.191:58428) 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45c3d07700 1 -- 172.21.15.191:0/2000508047 learned_addr learned my addr 172.21.15.191:0/2000508047 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45c3d07700 1 -- 172.21.15.191:0/2000508047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b0b60 con 0x55e218ddb400 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2000508047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55e218d7b3b0 tx=0x55e218ce1f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f1fdbdf4ce4e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45aacd5700 1 -- 172.21.15.191:0/2000508047 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218df8000 con 0x55e218ddb400 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45aacd5700 1 -- 172.21.15.191:0/2000508047 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218e02000 con 0x55e218ddb400 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.546+0000 7f45aacd5700 1 -- 172.21.15.191:0/2000508047 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218df8160 con 0x55e218ddb400 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- 172.21.15.191:0/2000508047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 msgr2=0x55e218c72680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2000508047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55e218d7b3b0 tx=0x55e218ce1f80 comp rx=0 tx=0).stop 2024-04-27T04:35:42.585 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- 172.21.15.191:0/2000508047 shutdown_connections 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2000508047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- 172.21.15.191:0/2000508047 >> 172.21.15.191:0/2000508047 conn(0x55e218ddb000 msgr2=0x55e218a76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- 172.21.15.191:0/2000508047 shutdown_connections 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- 172.21.15.191:0/2000508047 wait complete. 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 Processor -- start 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- start start 2024-04-27T04:35:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.547+0000 7f45c655ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e218a2a000 con 0x55e218ddb400 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c3d07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c3d07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58442/0 (socket says 172.21.15.191:58442) 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c3d07700 1 -- 172.21.15.191:0/2400788847 learned_addr learned my addr 172.21.15.191:0/2400788847 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c3d07700 1 -- 172.21.15.191:0/2400788847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2189b0d00 con 0x55e218ddb400 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c3d07700 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55e218dfa210 tx=0x55e218dbe4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45a94d2700 1 -- 172.21.15.191:0/2400788847 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218df82c0 con 0x55e218ddb400 2024-04-27T04:35:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45a94d2700 1 -- 172.21.15.191:0/2400788847 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e218e02180 con 0x55e218ddb400 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45a94d2700 1 -- 172.21.15.191:0/2400788847 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e218df8000 con 0x55e218ddb400 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2189b0340 con 0x55e218ddb400 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.548+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2189b0b60 con 0x55e218ddb400 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.549+0000 7f45a94d2700 1 -- 172.21.15.191:0/2400788847 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e21972bb80 con 0x55e218ddb400 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.549+0000 7f45a94d2700 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218e7c000 0x55e218c73700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.549+0000 7f45a94d2700 1 -- 172.21.15.191:0/2400788847 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e218a03500 con 0x55e218ddb400 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.549+0000 7f45c655ea80 5 note: GC not initialized 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.549+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218e7c000 0x55e218c73700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.550+0000 7f45c4d09700 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218e7c000 0x55e218c73700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55e218d7bd10 tx=0x55e218dbe360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.550+0000 7f45a6ccd700 20 reqs_thread_entry: start 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.550+0000 7f45c655ea80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.550+0000 7f45c655ea80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.550+0000 7f45a4cc9700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:12 2024 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45a44a7700 20 reqs_thread_entry: start 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb800 con 0x55e218989800 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda800 con 0x55e218d19400 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c400 con 0x55e218989800 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.551+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c800 con 0x55e218d19c00 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7cc00 con 0x55e218d19c00 2024-04-27T04:35:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d000 con 0x55e218d19c00 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d400 con 0x55e218989800 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d800 con 0x55e218989800 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.552+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7dc00 con 0x55e218d19c00 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.553+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.553+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c800 con 0x55e218d19c00 2024-04-27T04:35:42.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.553+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.553+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7cc00 con 0x55e218d19000 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda400 con 0x55e218989800 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddac00 con 0x55e218d19400 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddbc00 con 0x55e218d19c00 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.554+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb800 con 0x55e218d19000 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda400 con 0x55e218d19000 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c400 con 0x55e218d19c00 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d400 con 0x55e218d19000 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.555+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d800 con 0x55e218989800 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7cc00 con 0x55e218d19c00 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d000 con 0x55e218d19000 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7dc00 con 0x55e218d19400 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.556+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c800 con 0x55e218d19000 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c400 con 0x55e218989800 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddbc00 con 0x55e218989800 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d400 con 0x55e218d19c00 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d800 con 0x55e218989800 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.557+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d000 con 0x55e218989800 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c800 con 0x55e218d19c00 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c400 con 0x55e218d19c00 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7dc00 con 0x55e218d19c00 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddb800 con 0x55e218d19000 2024-04-27T04:35:42.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.558+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.598 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:42.598 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:42.598 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.598 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "anonymous", 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "anonymous", 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_buckets", 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 231, 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.599 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.600 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_buckets", 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 322, 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.601 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 5 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.602 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:42.603 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.604 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.604 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.604 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.604 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.604 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_acls", 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 1026, 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.605 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_obj", 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 4, 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_bucket", 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 515, 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.606 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_acls", 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.607 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_obj", 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.608 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.609 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.610 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.611 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.613 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.613 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.613 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.613 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.613 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.613 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.614 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.614 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.614 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.614 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.614 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:42.614 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.615 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.616 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "anonymous", 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.617 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_buckets", 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 231, 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:42.618 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 231, 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.619 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 8, 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 8 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.620 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.621 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.621 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:42.621 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.621 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:42.621 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.622 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.622 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:42.622 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.622 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.622 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.622 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_acls", 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 1026, 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.624 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_obj", 2024-04-27T04:35:42.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 4, 2024-04-27T04:35:42.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.624 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.624 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.624 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.559+0000 7f45ab4d6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.559+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94429568196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda400 con 0x55e218989800 2024-04-27T04:35:42.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.560+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94429568196608] v33'205 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.560+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94429566630656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddbc00 con 0x55e218d19400 2024-04-27T04:35:42.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.561+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94429566630656] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.561+0000 7f45c655ea80 20 remove_watcher() i=0 2024-04-27T04:35:42.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.561+0000 7f45c655ea80 2 removed watcher, disabling cache 2024-04-27T04:35:42.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.561+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94429566631808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218dda800 con 0x55e218989800 2024-04-27T04:35:42.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.562+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94429566631808] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.563+0000 7f45c655ea80 20 remove_watcher() i=1 2024-04-27T04:35:42.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.563+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94429566632960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddac00 con 0x55e218d19c00 2024-04-27T04:35:42.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.564+0000 7f45c4508700 1 -- 172.21.15.191:0/2793448244 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94429566632960] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd200 con 0x55e218d19c00 2024-04-27T04:35:42.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.564+0000 7f45c655ea80 20 remove_watcher() i=2 2024-04-27T04:35:42.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.564+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94429566634112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218989c00 con 0x55e218d19400 2024-04-27T04:35:42.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.564+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94429566634112] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dcd440 con 0x55e218d19400 2024-04-27T04:35:42.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.565+0000 7f45c655ea80 20 remove_watcher() i=3 2024-04-27T04:35:42.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.565+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94429566635264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218ddac00 con 0x55e218989800 2024-04-27T04:35:42.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.566+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94429566635264] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218989800 2024-04-27T04:35:42.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.566+0000 7f45c655ea80 20 remove_watcher() i=4 2024-04-27T04:35:42.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.566+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94429566636416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7cc00 con 0x55e218d19000 2024-04-27T04:35:42.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.567+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94429566636416] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.567+0000 7f45c655ea80 20 remove_watcher() i=5 2024-04-27T04:35:42.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.567+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94429566637568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7d400 con 0x55e218d19000 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.568+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94429566637568] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.568+0000 7f45c655ea80 20 remove_watcher() i=6 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.568+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94429566638720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e218e7c800 con 0x55e218d19000 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c4d09700 1 -- 172.21.15.191:0/2793448244 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94429566638720] v33'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e218dccb40 con 0x55e218d19000 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 20 remove_watcher() i=7 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e218d19000 msgr2=0x55e218c73180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55e218d19000 0x55e218c73180 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e218989800 msgr2=0x55e218c71b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55e218989800 0x55e218c71b80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e218d19400 msgr2=0x55e218c72c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55e218d19400 0x55e218c72c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e218d19c00 msgr2=0x55e218c72100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55e218d19c00 0x55e218c72100 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18c00 msgr2=0x55e218c71600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18c00 0x55e218c71600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e218d1b770 tx=0x55e218dbe0c0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 msgr2=0x55e218c70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2793448244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218989400 0x55e218c70b00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55e21981e0f0 tx=0x55e218ce05a0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 reap_dead start 2024-04-27T04:35:42.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c3d07700 1 -- 172.21.15.191:0/2793448244 reap_dead start 2024-04-27T04:35:42.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 shutdown_connections 2024-04-27T04:35:42.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 >> 172.21.15.191:0/2793448244 conn(0x55e218988c00 msgr2=0x55e218a75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 shutdown_connections 2024-04-27T04:35:42.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2793448244 wait complete. 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218e7c000 msgr2=0x55e218c73700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218e7c000 0x55e218c73700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55e218d7bd10 tx=0x55e218dbe360 comp rx=0 tx=0).stop 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 msgr2=0x55e218c72680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.569+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55e218dfa210 tx=0x55e218dbe4e0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 shutdown_connections 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218e7c000 0x55e218c73700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 --2- 172.21.15.191:0/2400788847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218ddb400 0x55e218c72680 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 >> 172.21.15.191:0/2400788847 conn(0x55e218ddb000 msgr2=0x55e218a76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 shutdown_connections 2024-04-27T04:35:42.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/2400788847 wait complete. 2024-04-27T04:35:42.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18000 msgr2=0x55e218c71080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18000 0x55e218c71080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e21981ef60 tx=0x55e218ce0c60 comp rx=0 tx=0).stop 2024-04-27T04:35:42.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 msgr2=0x55e218c70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55e2197535f0 tx=0x55e218a79920 comp rx=0 tx=0).stop 2024-04-27T04:35:42.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 shutdown_connections 2024-04-27T04:35:42.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55e218d18000 0x55e218c71080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 --2- 172.21.15.191:0/3880159122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e218988800 0x55e218c70580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 >> 172.21.15.191:0/3880159122 conn(0x55e218988400 msgr2=0x55e218a75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 shutdown_connections 2024-04-27T04:35:42.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.570+0000 7f45c655ea80 1 -- 172.21.15.191:0/3880159122 wait complete. 2024-04-27T04:35:42.645 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.645 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_bucket", 2024-04-27T04:35:42.645 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 515, 2024-04-27T04:35:42.645 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.645 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_buckets", 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 322, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_acls", 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.646 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_obj", 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 2103, 2024-04-27T04:35:42.647 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:42.648 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 35, 2024-04-27T04:35:42.648 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 34 2024-04-27T04:35:42.648 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.648 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.648 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.648 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:42.648 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-04-27T04:35:42.649 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-27T04:35:42.649 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-27T04:35:42.649 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-27T04:35:42.730 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:42.731 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f71280bca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f71280bca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5580a4933500 con 0x5580a4608c00 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f71280bca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5580a46aaa00 con 0x5580a4608800 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7125865700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5580a4608c00 0x5580a46f4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52632/0 (socket says 172.21.15.191:52632) 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7125865700 1 -- 172.21.15.191:0/4115192851 learned_addr learned my addr 172.21.15.191:0/4115192851 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7126066700 1 --2- 172.21.15.191:0/4115192851 >> v2:172.21.15.138:3300/0 conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7125064700 1 -- 172.21.15.191:0/4115192851 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003079089 0 0) 0x5580a53d6000 con 0x5580a4608c00 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7125064700 1 -- 172.21.15.191:0/4115192851 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x5580a4933680 con 0x5580a4608c00 2024-04-27T04:35:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7126066700 1 -- 172.21.15.191:0/4115192851 >> v1:172.21.15.138:6789/0 conn(0x5580a4608c00 legacy=0x5580a46f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.729+0000 7f7126066700 1 -- 172.21.15.191:0/4115192851 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4631d40 con 0x5580a4608800 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f7126066700 1 --2- 172.21.15.191:0/4115192851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x5580a48eb9e0 tx=0x5580a46f9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97fb482784cc748 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f7125064700 1 -- 172.21.15.191:0/4115192851 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c14a0 con 0x5580a4608800 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f7125064700 1 -- 172.21.15.191:0/4115192851 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a4933500 con 0x5580a4608800 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f7125064700 1 -- 172.21.15.191:0/4115192851 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c1600 con 0x5580a4608800 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 -- 172.21.15.191:0/4115192851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 msgr2=0x5580a48f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 --2- 172.21.15.191:0/4115192851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x5580a48eb9e0 tx=0x5580a46f9980 comp rx=0 tx=0).stop 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 -- 172.21.15.191:0/4115192851 shutdown_connections 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 --2- 172.21.15.191:0/4115192851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 -- 172.21.15.191:0/4115192851 >> 172.21.15.191:0/4115192851 conn(0x5580a4608400 msgr2=0x5580a46f4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 -- 172.21.15.191:0/4115192851 shutdown_connections 2024-04-27T04:35:42.745 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.730+0000 7f71280bca80 1 -- 172.21.15.191:0/4115192851 wait complete. 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f71280bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f71280bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a46aa500 con 0x5580a4608800 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f7126066700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f7126066700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58454/0 (socket says 172.21.15.191:58454) 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f7126066700 1 -- 172.21.15.191:0/3785785662 learned_addr learned my addr 172.21.15.191:0/3785785662 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f7126066700 1 -- 172.21.15.191:0/3785785662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4631ba0 con 0x5580a4608800 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.731+0000 7f7126066700 1 --2- 172.21.15.191:0/3785785662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5580a53d9800 tx=0x5580a46f9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5226264348909c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f7124863700 1 -- 172.21.15.191:0/3785785662 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c1760 con 0x5580a4608800 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f7124863700 1 -- 172.21.15.191:0/3785785662 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a4933980 con 0x5580a4608800 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f7124863700 1 -- 172.21.15.191:0/3785785662 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c11e0 con 0x5580a4608800 2024-04-27T04:35:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 -- 172.21.15.191:0/3785785662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 msgr2=0x5580a48f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 --2- 172.21.15.191:0/3785785662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5580a53d9800 tx=0x5580a46f9800 comp rx=0 tx=0).stop 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 -- 172.21.15.191:0/3785785662 shutdown_connections 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 --2- 172.21.15.191:0/3785785662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 -- 172.21.15.191:0/3785785662 >> 172.21.15.191:0/3785785662 conn(0x5580a4608400 msgr2=0x5580a46f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 -- 172.21.15.191:0/3785785662 shutdown_connections 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 -- 172.21.15.191:0/3785785662 wait complete. 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.732+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f71280bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f71280bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a46aa3c0 con 0x5580a4608800 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7126066700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7126066700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58460/0 (socket says 172.21.15.191:58460) 2024-04-27T04:35:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7126066700 1 -- 172.21.15.191:0/2545054535 learned_addr learned my addr 172.21.15.191:0/2545054535 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7126066700 1 -- 172.21.15.191:0/2545054535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4631a00 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7126066700 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x5580a53d99e0 tx=0x5580a46f9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7123060700 1 -- 172.21.15.191:0/2545054535 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c14a0 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7123060700 1 -- 172.21.15.191:0/2545054535 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a4933b00 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5580a4631860 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f7123060700 1 -- 172.21.15.191:0/2545054535 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c1600 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.733+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580a46316c0 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f71280bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f71280bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a46aa000 con 0x5580a4609400 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7123060700 1 -- 172.21.15.191:0/2545054535 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5580a53aab00 con 0x5580a4608800 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7126867700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7126867700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58472/0 (socket says 172.21.15.191:58472) 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7126867700 1 -- 172.21.15.191:0/1309381913 learned_addr learned my addr 172.21.15.191:0/1309381913 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7123060700 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994000 0x5580a48f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7123060700 1 -- 172.21.15.191:0/2545054535 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5580a4683180 con 0x5580a4608800 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7126066700 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994000 0x5580a48f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.734+0000 7f7126867700 1 -- 172.21.15.191:0/1309381913 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4630000 con 0x5580a4609400 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f7126867700 1 --2- 172.21.15.191:0/1309381913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x5580a4996060 tx=0x5580a495cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e65db0448ac6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f7126066700 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994000 0x5580a48f1080 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5580a549b020 tx=0x5580a495cc60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f712085b700 1 -- 172.21.15.191:0/1309381913 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c14a0 con 0x5580a4609400 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f712085b700 1 -- 172.21.15.191:0/1309381913 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a46a9c80 con 0x5580a4609400 2024-04-27T04:35:42.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f712085b700 1 -- 172.21.15.191:0/1309381913 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c1600 con 0x5580a4609400 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 -- 172.21.15.191:0/1309381913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 msgr2=0x5580a48f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 --2- 172.21.15.191:0/1309381913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x5580a4996060 tx=0x5580a495cb40 comp rx=0 tx=0).stop 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 -- 172.21.15.191:0/1309381913 shutdown_connections 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 --2- 172.21.15.191:0/1309381913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 -- 172.21.15.191:0/1309381913 >> 172.21.15.191:0/1309381913 conn(0x5580a4608c00 msgr2=0x5580a46f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 -- 172.21.15.191:0/1309381913 shutdown_connections 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 -- 172.21.15.191:0/1309381913 wait complete. 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.735+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f71280bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f71280bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a4644640 con 0x5580a4609400 2024-04-27T04:35:42.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f7126867700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f7126867700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58486/0 (socket says 172.21.15.191:58486) 2024-04-27T04:35:42.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 learned_addr learned my addr 172.21.15.191:0/4207944208 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4630ea0 con 0x5580a4609400 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f7126867700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x5580a53d99b0 tx=0x5580a495c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f711f058700 1 -- 172.21.15.191:0/4207944208 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c1760 con 0x5580a4609400 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f711f058700 1 -- 172.21.15.191:0/4207944208 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a46a9b00 con 0x5580a4609400 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5580a4630820 con 0x5580a4609400 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f711f058700 1 -- 172.21.15.191:0/4207944208 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a46c14a0 con 0x5580a4609400 2024-04-27T04:35:42.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.736+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580a46309c0 con 0x5580a4609400 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.737+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.737+0000 7f711f058700 1 -- 172.21.15.191:0/4207944208 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5580a53aadc0 con 0x5580a4609400 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.737+0000 7f711f058700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994c00 0x5580a48f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f711f058700 1 -- 172.21.15.191:0/4207944208 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5580a4683340 con 0x5580a4609400 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f7125865700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994c00 0x5580a48f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5580a4609800 0x5580a48f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4609c00 con 0x5580a4609800 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f7126867700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5580a4609800 0x5580a48f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f7125865700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994c00 0x5580a48f1600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5580a53d8030 tx=0x5580a4a4e0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f7126867700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5580a4609800 0x5580a48f1b80 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f71280bca80 20 realm 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.738+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995000 con 0x5580a4609800 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995400 con 0x5580a4609800 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995800 con 0x5580a4609800 2024-04-27T04:35:42.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5580a4995c00 0x5580a48f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995000 con 0x5580a4995c00 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f7126066700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5580a4995c00 0x5580a48f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.739+0000 7f7126066700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5580a4995c00 0x5580a48f2100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f71280bca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f71280bca80 20 searching for the correct realm 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995400 con 0x5580a4995c00 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f7126066700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5580a4995400 0x5580a48f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995000 con 0x5580a4995400 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.740+0000 7f7125865700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5580a4995400 0x5580a48f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.741+0000 7f7125865700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5580a4995400 0x5580a48f2c00 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.741+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.741+0000 7f7125865700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5580a4995000 0x5580a48f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.741+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.741+0000 7f7126867700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5580a4995000 0x5580a48f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.741+0000 7f7126867700 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5580a4995000 0x5580a48f3180 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.742+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995400 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.743+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995400 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.744+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.745+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995400 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.746+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995400 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995400 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.784 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.747+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995c00 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995400 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f71280bca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f71280bca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f71280bca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.748+0000 7f71280bca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4609c00 con 0x5580a4609800 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60400 con 0x5580a4609800 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60800 con 0x5580a4995c00 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 zone default found 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 4 Realm: () 2024-04-27T04:35:42.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 zonegroup default 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.749+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60c00 con 0x5580a4995000 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61000 con 0x5580a4995c00 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61400 con 0x5580a4995c00 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 started sync module instance, tier type = 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.750+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61800 con 0x5580a4995400 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.751+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.751+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94011004975872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60800 con 0x5580a4995400 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.751+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61000 con 0x5580a4609800 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.752+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94011004975872] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.752+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.752+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94011004977024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61400 con 0x5580a4609800 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.752+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60400 con 0x5580a4995c00 2024-04-27T04:35:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.753+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94011004977024] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.753+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.754+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94011004978176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60c00 con 0x5580a4995c00 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.754+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4609c00 con 0x5580a4995400 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.755+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.755+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94011004979328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4995800 con 0x5580a4995400 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.755+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4609800 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.755+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94011004978176] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.756+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94011004979328] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.756+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.756+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94011004980480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61800 con 0x5580a4609800 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.756+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60800 con 0x5580a4995000 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.757+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.757+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94011004981632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61c00 con 0x5580a4995000 2024-04-27T04:35:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.757+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61000 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.757+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94011004980480] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.758+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94011004981632] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.758+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.758+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94011004982784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61400 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.758+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60000 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(58 notify.7 [create] v33'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94011004982784] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94011004983936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61800 con 0x5580a4995000 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=0 2024-04-27T04:35:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=1 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=2 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=3 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=4 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=5 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.759+0000 7f71280bca80 20 add_watcher() i=6 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.760+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94011004983936] v33'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.760+0000 7f71280bca80 20 add_watcher() i=7 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.760+0000 7f71280bca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.760+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60800 con 0x5580a4609800 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.760+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.760+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94011006566400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61c00 con 0x5580a4609800 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.761+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94011006566400] v33'206 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.761+0000 7f710d034700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.761+0000 7f71280bca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.761+0000 7f71280bca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.761+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f71280bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f71280bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a4644780 con 0x5580a4a61400 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f7125865700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f7125865700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58496/0 (socket says 172.21.15.191:58496) 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f7125865700 1 -- 172.21.15.191:0/4057004246 learned_addr learned my addr 172.21.15.191:0/4057004246 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f7125865700 1 -- 172.21.15.191:0/4057004246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4630b60 con 0x5580a4a61400 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.762+0000 7f7125865700 1 --2- 172.21.15.191:0/4057004246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5580a4997650 tx=0x5580a495df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1492823aed39c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f710c833700 1 -- 172.21.15.191:0/4057004246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a4a8a000 con 0x5580a4a61400 2024-04-27T04:35:42.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f710c833700 1 -- 172.21.15.191:0/4057004246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a4a82180 con 0x5580a4a61400 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f710c833700 1 -- 172.21.15.191:0/4057004246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a4a8a160 con 0x5580a4a61400 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 -- 172.21.15.191:0/4057004246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 msgr2=0x5580a48f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 --2- 172.21.15.191:0/4057004246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5580a4997650 tx=0x5580a495df80 comp rx=0 tx=0).stop 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 -- 172.21.15.191:0/4057004246 shutdown_connections 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 --2- 172.21.15.191:0/4057004246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 -- 172.21.15.191:0/4057004246 >> 172.21.15.191:0/4057004246 conn(0x5580a4a61000 msgr2=0x5580a46f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 -- 172.21.15.191:0/4057004246 shutdown_connections 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.763+0000 7f71280bca80 1 -- 172.21.15.191:0/4057004246 wait complete. 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f71280bca80 1 Processor -- start 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f71280bca80 1 -- start start 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f71280bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f71280bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a46aa000 con 0x5580a4a61400 2024-04-27T04:35:42.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f7125865700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f7125865700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58512/0 (socket says 172.21.15.191:58512) 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.764+0000 7f7125865700 1 -- 172.21.15.191:0/383609046 learned_addr learned my addr 172.21.15.191:0/383609046 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f7125865700 1 -- 172.21.15.191:0/383609046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a4630d00 con 0x5580a4a61400 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f7125865700 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5580a4a8d200 tx=0x5580a4a4e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f710b030700 1 -- 172.21.15.191:0/383609046 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a4a8a2c0 con 0x5580a4a61400 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f710b030700 1 -- 172.21.15.191:0/383609046 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580a4a82300 con 0x5580a4a61400 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5580a4630340 con 0x5580a4a61400 2024-04-27T04:35:42.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f710b030700 1 -- 172.21.15.191:0/383609046 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580a4a8a000 con 0x5580a4a61400 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.765+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580a4630b60 con 0x5580a4a61400 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.766+0000 7f710b030700 1 -- 172.21.15.191:0/383609046 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5580a53abb80 con 0x5580a4a61400 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.766+0000 7f710b030700 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4b0a000 0x5580a48f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.766+0000 7f710b030700 1 -- 172.21.15.191:0/383609046 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5580a4683500 con 0x5580a4a61400 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.766+0000 7f71280bca80 5 note: GC not initialized 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.766+0000 7f7126867700 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4b0a000 0x5580a48f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.766+0000 7f7126867700 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4b0a000 0x5580a48f3700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5580a4997f80 tx=0x5580a4a4e360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f710882b700 20 reqs_thread_entry: start 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f71280bca80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f71280bca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f7106827700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:12 2024 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f7106005700 20 reqs_thread_entry: start 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f71280bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.767+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a61800 con 0x5580a4609800 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.768+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.768+0000 7f71280bca80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.768+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4a60800 con 0x5580a4995000 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.768+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:42.796 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_buckets", 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 322, 2024-04-27T04:35:42.797 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.798 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 5 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.799 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_acls", 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 1026, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.800 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_obj", 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 4, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_bucket", 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 515, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.801 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_acls", 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_obj", 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:42.802 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.803 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.769+0000 7f710d034700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.769+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94011006566400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0a400 con 0x5580a4609800 2024-04-27T04:35:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.770+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94011006566400] v33'207 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.770+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94011004975872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0a800 con 0x5580a4995400 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.771+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94011004975872] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.771+0000 7f71280bca80 20 remove_watcher() i=0 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.771+0000 7f71280bca80 2 removed watcher, disabling cache 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.771+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94011004977024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0ac00 con 0x5580a4609800 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.772+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94011004977024] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.772+0000 7f71280bca80 20 remove_watcher() i=1 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.772+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94011004978176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0b000 con 0x5580a4995c00 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.773+0000 7f7126066700 1 -- 172.21.15.191:0/4207944208 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94011004978176] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d200 con 0x5580a4995c00 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.773+0000 7f71280bca80 20 remove_watcher() i=2 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.773+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94011004979328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0b400 con 0x5580a4995400 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.774+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94011004979328] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3d440 con 0x5580a4995400 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.774+0000 7f71280bca80 20 remove_watcher() i=3 2024-04-27T04:35:42.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.774+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94011004980480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0b800 con 0x5580a4609800 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.775+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94011004980480] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4609800 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.775+0000 7f71280bca80 20 remove_watcher() i=4 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.775+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94011004981632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0bc00 con 0x5580a4995000 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.776+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94011004981632] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.776+0000 7f71280bca80 20 remove_watcher() i=5 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.776+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94011004982784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0a400 con 0x5580a4995000 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.777+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94011004982784] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.777+0000 7f71280bca80 20 remove_watcher() i=6 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.777+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94011004983936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5580a4b0ac00 con 0x5580a4995000 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f7126867700 1 -- 172.21.15.191:0/4207944208 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94011004983936] v33'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580a4a3cb40 con 0x5580a4995000 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 20 remove_watcher() i=7 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5580a4995000 msgr2=0x5580a48f3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5580a4995000 0x5580a48f3180 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5580a4609800 msgr2=0x5580a48f1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5580a4609800 0x5580a48f1b80 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5580a4995400 msgr2=0x5580a48f2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5580a4995400 0x5580a48f2c00 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5580a4995c00 msgr2=0x5580a48f2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5580a4995c00 0x5580a48f2100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994c00 msgr2=0x5580a48f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994c00 0x5580a48f1600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5580a53d8030 tx=0x5580a4a4e0c0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 msgr2=0x5580a48f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f71280bca80 1 --2- 172.21.15.191:0/4207944208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4609400 0x5580a48f0b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x5580a53d99b0 tx=0x5580a495c5a0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.778+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 reap_dead start 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f7125865700 1 -- 172.21.15.191:0/4207944208 reap_dead start 2024-04-27T04:35:42.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 shutdown_connections 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 >> 172.21.15.191:0/4207944208 conn(0x5580a4608c00 msgr2=0x5580a46f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 shutdown_connections 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/4207944208 wait complete. 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4b0a000 msgr2=0x5580a48f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4b0a000 0x5580a48f3700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5580a4997f80 tx=0x5580a4a4e360 comp rx=0 tx=0).stop 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 msgr2=0x5580a48f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5580a4a8d200 tx=0x5580a4a4e4e0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 shutdown_connections 2024-04-27T04:35:42.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4b0a000 0x5580a48f3700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/383609046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4a61400 0x5580a48f2680 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 >> 172.21.15.191:0/383609046 conn(0x5580a4a61000 msgr2=0x5580a46f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 shutdown_connections 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/383609046 wait complete. 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994000 msgr2=0x5580a48f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994000 0x5580a48f1080 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5580a549b020 tx=0x5580a495cc60 comp rx=0 tx=0).stop 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 msgr2=0x5580a48f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x5580a53d99e0 tx=0x5580a46f9980 comp rx=0 tx=0).stop 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 shutdown_connections 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5580a4994000 0x5580a48f1080 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 --2- 172.21.15.191:0/2545054535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580a4608800 0x5580a48f0580 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 >> 172.21.15.191:0/2545054535 conn(0x5580a4608400 msgr2=0x5580a46f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 shutdown_connections 2024-04-27T04:35:42.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.779+0000 7f71280bca80 1 -- 172.21.15.191:0/2545054535 wait complete. 2024-04-27T04:35:42.816 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:42.816 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.817 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.818 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.819 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 8, 2024-04-27T04:35:42.820 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 8 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.821 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_acls", 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 1026, 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.822 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "get_obj", 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 4, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_bucket", 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 515, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.823 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "list_buckets", 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 322, 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_acls", 2024-04-27T04:35:42.824 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 2 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_obj", 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:42.825 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 2103, 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 35, 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 34 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.826 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:42.826 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-04-27T04:35:42.827 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-27T04:35:42.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-27T04:35:42.827 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-04-27T04:35:42.874 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:42.874 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.872+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.872+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.872+0000 7ffa4d1a7a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5641ef85cc00 0x5641efb44580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.872+0000 7ffa4d1a7a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5641efb87500 con 0x5641ef85c800 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.872+0000 7ffa4d1a7a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x5641ef8fea00 con 0x5641ef85cc00 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4b151700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x5641ef85c800 0x5641ef948800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52636/0 (socket says 172.21.15.191:52636) 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a950700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x5641ef85cc00 0x5641efb44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4b151700 1 -- 172.21.15.191:0/773936104 learned_addr learned my addr 172.21.15.191:0/773936104 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:42.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a14f700 1 -- 172.21.15.191:0/773936104 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807778337 0 0) 0x5641f062a000 con 0x5641ef85c800 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a14f700 1 -- 172.21.15.191:0/773936104 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x5641efb87680 con 0x5641ef85c800 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a950700 1 -- 172.21.15.191:0/773936104 >> v1:172.21.15.138:6789/0 conn(0x5641ef85c800 legacy=0x5641ef948800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a950700 1 -- 172.21.15.191:0/773936104 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef885d40 con 0x5641ef85cc00 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a950700 1 --2- 172.21.15.191:0/773936104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85cc00 0x5641efb44580 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5641efb3f890 tx=0x5641ef94d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4466141a01cc6a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a14f700 1 -- 172.21.15.191:0/773936104 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9154a0 con 0x5641ef85cc00 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a14f700 1 -- 172.21.15.191:0/773936104 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641efb87500 con 0x5641ef85cc00 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.873+0000 7ffa4a14f700 1 -- 172.21.15.191:0/773936104 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef915600 con 0x5641ef85cc00 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/773936104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85cc00 msgr2=0x5641efb44580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/773936104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85cc00 0x5641efb44580 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5641efb3f890 tx=0x5641ef94d980 comp rx=0 tx=0).stop 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/773936104 shutdown_connections 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/773936104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85cc00 0x5641efb44580 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/773936104 >> 172.21.15.191:0/773936104 conn(0x5641ef85c400 msgr2=0x5641ef948000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/773936104 shutdown_connections 2024-04-27T04:35:42.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.874+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/773936104 wait complete. 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4d1a7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4d1a7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ef8fe500 con 0x5641ef85c800 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4b151700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4b151700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58530/0 (socket says 172.21.15.191:58530) 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4b151700 1 -- 172.21.15.191:0/1476731761 learned_addr learned my addr 172.21.15.191:0/1476731761 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4b151700 1 -- 172.21.15.191:0/1476731761 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef885ba0 con 0x5641ef85c800 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4b151700 1 --2- 172.21.15.191:0/1476731761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x5641f062d770 tx=0x5641ef94db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04ba45ccafc2df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4994e700 1 -- 172.21.15.191:0/1476731761 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9154a0 con 0x5641ef85c800 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4994e700 1 -- 172.21.15.191:0/1476731761 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641efb87b00 con 0x5641ef85c800 2024-04-27T04:35:42.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.875+0000 7ffa4994e700 1 -- 172.21.15.191:0/1476731761 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef915600 con 0x5641ef85c800 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1476731761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 msgr2=0x5641efb44580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1476731761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x5641f062d770 tx=0x5641ef94db60 comp rx=0 tx=0).stop 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1476731761 shutdown_connections 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1476731761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1476731761 >> 172.21.15.191:0/1476731761 conn(0x5641ef85c400 msgr2=0x5641ef949000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1476731761 shutdown_connections 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1476731761 wait complete. 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4d1a7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ef8fe3c0 con 0x5641ef85c800 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4b151700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4b151700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58542/0 (socket says 172.21.15.191:58542) 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.876+0000 7ffa4b151700 1 -- 172.21.15.191:0/2239699932 learned_addr learned my addr 172.21.15.191:0/2239699932 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4b151700 1 -- 172.21.15.191:0/2239699932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef885a00 con 0x5641ef85c800 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4b151700 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5641f062de30 tx=0x5641ef94d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4814b700 1 -- 172.21.15.191:0/2239699932 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9158c0 con 0x5641ef85c800 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4814b700 1 -- 172.21.15.191:0/2239699932 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641ef8fdc80 con 0x5641ef85c800 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5641ef885860 con 0x5641ef85c800 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4814b700 1 -- 172.21.15.191:0/2239699932 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9154a0 con 0x5641ef85c800 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.877+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641ef8856c0 con 0x5641ef85c800 2024-04-27T04:35:42.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4814b700 1 -- 172.21.15.191:0/2239699932 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5641f05feb00 con 0x5641ef85c800 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4d1a7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4d1a7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ef8fe000 con 0x5641ef85d400 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4814b700 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbea000 0x5641efb45080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4814b700 1 -- 172.21.15.191:0/2239699932 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5641ef8d7180 con 0x5641ef85c800 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b952700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b952700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58554/0 (socket says 172.21.15.191:58554) 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b952700 1 -- 172.21.15.191:0/3065308424 learned_addr learned my addr 172.21.15.191:0/3065308424 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b151700 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbea000 0x5641efb45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b952700 1 -- 172.21.15.191:0/3065308424 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef884000 con 0x5641ef85d400 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b151700 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbea000 0x5641efb45080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5641f06eaf00 tx=0x5641efba6240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.913 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa4b952700 1 --2- 172.21.15.191:0/3065308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5641f06eae10 tx=0x5641efba70e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d158fb050d76be server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.878+0000 7ffa45946700 1 -- 172.21.15.191:0/3065308424 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9158c0 con 0x5641ef85d400 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa45946700 1 -- 172.21.15.191:0/3065308424 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641efb86480 con 0x5641ef85d400 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa45946700 1 -- 172.21.15.191:0/3065308424 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9154a0 con 0x5641ef85d400 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/3065308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 msgr2=0x5641efb44b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/3065308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5641f06eae10 tx=0x5641efba70e0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/3065308424 shutdown_connections 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/3065308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/3065308424 >> 172.21.15.191:0/3065308424 conn(0x5641ef85cc00 msgr2=0x5641ef949800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/3065308424 shutdown_connections 2024-04-27T04:35:42.914 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/3065308424 wait complete. 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.879+0000 7ffa4d1a7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ef898640 con 0x5641ef85d400 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4b952700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4b952700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58556/0 (socket says 172.21.15.191:58556) 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 learned_addr learned my addr 172.21.15.191:0/689875338 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef884ea0 con 0x5641ef85d400 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4b952700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5641f06ea420 tx=0x5641efba6d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa44143700 1 -- 172.21.15.191:0/689875338 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef915b80 con 0x5641ef85d400 2024-04-27T04:35:42.915 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa44143700 1 -- 172.21.15.191:0/689875338 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641efb86180 con 0x5641ef85d400 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5641ef884820 con 0x5641ef85d400 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa44143700 1 -- 172.21.15.191:0/689875338 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef9158c0 con 0x5641ef85d400 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.880+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641ef8849c0 con 0x5641ef85d400 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa44143700 1 -- 172.21.15.191:0/689875338 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5641f05fedc0 con 0x5641ef85d400 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa44143700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbeac00 0x5641efb45600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa44143700 1 -- 172.21.15.191:0/689875338 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5641ef8d7340 con 0x5641ef85d400 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa4a950700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbeac00 0x5641efb45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5641ef85d800 0x5641efb45b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641ef85dc00 con 0x5641ef85d800 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.881+0000 7ffa4b952700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5641ef85d800 0x5641efb45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.916 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4a950700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbeac00 0x5641efb45600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5641efb3f890 tx=0x5641ef94d800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4b952700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5641ef85d800 0x5641efb45b80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 realm 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb000 con 0x5641ef85d800 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.917 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb400 con 0x5641ef85d800 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb800 con 0x5641ef85d800 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641efbebc00 0x5641efb46100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.882+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb000 con 0x5641efbebc00 2024-04-27T04:35:42.918 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4b151700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641efbebc00 0x5641efb46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4b151700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641efbebc00 0x5641efb46100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4d1a7a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4d1a7a80 20 searching for the correct realm 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb400 con 0x5641efbebc00 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4b151700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641efbeb400 0x5641efb46c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.883+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb000 con 0x5641efbeb400 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.884+0000 7ffa4a950700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641efbeb400 0x5641efb46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.884+0000 7ffa4a950700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641efbeb400 0x5641efb46c00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.884+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.884+0000 7ffa4a950700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5641efbeb000 0x5641efb47180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.884+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.884+0000 7ffa4b952700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5641efbeb000 0x5641efb47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5641efbeb000 0x5641efb47180 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.885+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb400 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.886+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb400 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.887+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.888+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb400 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.889+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb400 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.890+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb400 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbebc00 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb400 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4d1a7a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4d1a7a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:42.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4d1a7a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4d1a7a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.891+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641ef85dc00 con 0x5641ef85d800 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0400 con 0x5641ef85d800 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0800 con 0x5641efbebc00 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 zone default found 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 4 Realm: () 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 zonegroup default 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.892+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0c00 con 0x5641efbeb000 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1000 con 0x5641efbebc00 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1400 con 0x5641efbebc00 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 started sync module instance, tier type = 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:42.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.893+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1800 con 0x5641efbeb400 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.894+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.894+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94841194396416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0800 con 0x5641efbeb400 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.894+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1000 con 0x5641ef85d800 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.895+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94841194396416] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.895+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.895+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94841194397568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1400 con 0x5641ef85d800 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.895+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0400 con 0x5641efbebc00 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.896+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.896+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94841194398720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0c00 con 0x5641efbebc00 2024-04-27T04:35:42.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.897+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641ef85dc00 con 0x5641efbeb400 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.897+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94841194397568] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.897+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.898+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94841194399872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efbeb800 con 0x5641efbeb400 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.898+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641ef85d800 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.898+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94841194398720] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.899+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.899+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94841194401024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1800 con 0x5641ef85d800 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.899+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94841194399872] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.899+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0800 con 0x5641efbeb000 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.900+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.900+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94841194402176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1c00 con 0x5641efbeb000 2024-04-27T04:35:42.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.900+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1000 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.900+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94841194401024] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.900+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(56 notify.6 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.901+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94841194403328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1400 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.901+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0000 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.901+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94841194402176] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.901+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94841194403328] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94841194404480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1800 con 0x5641efbeb000 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=0 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=1 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=2 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=3 2024-04-27T04:35:42.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=4 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=5 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=6 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94841194404480] v33'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 20 add_watcher() i=7 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.902+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0800 con 0x5641ef85d800 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.903+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.903+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94841195970560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1c00 con 0x5641ef85d800 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94841195970560] v33'208 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa3211f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4d1a7a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4d1a7a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:42.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4d1a7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.904+0000 7ffa4d1a7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ef898780 con 0x5641efcb1400 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa4a950700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa4a950700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58570/0 (socket says 172.21.15.191:58570) 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa4a950700 1 -- 172.21.15.191:0/1047637512 learned_addr learned my addr 172.21.15.191:0/1047637512 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa4a950700 1 -- 172.21.15.191:0/1047637512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef884b60 con 0x5641efcb1400 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa4a950700 1 --2- 172.21.15.191:0/1047637512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5641efc51800 tx=0x5641ef94d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4818bf2781251f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa3191e700 1 -- 172.21.15.191:0/1047637512 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641ef915760 con 0x5641efcb1400 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa3191e700 1 -- 172.21.15.191:0/1047637512 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641efcd4000 con 0x5641efcb1400 2024-04-27T04:35:42.932 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.905+0000 7ffa3191e700 1 -- 172.21.15.191:0/1047637512 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641efcd6000 con 0x5641efcb1400 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1047637512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 msgr2=0x5641efb46680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1047637512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5641efc51800 tx=0x5641ef94d9e0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1047637512 shutdown_connections 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1047637512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1047637512 >> 172.21.15.191:0/1047637512 conn(0x5641efcb1000 msgr2=0x5641ef94a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1047637512 shutdown_connections 2024-04-27T04:35:42.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1047637512 wait complete. 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 Processor -- start 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- start start 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.906+0000 7ffa4d1a7a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ef8fe000 con 0x5641efcb1400 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4a950700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4a950700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58582/0 (socket says 172.21.15.191:58582) 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4a950700 1 -- 172.21.15.191:0/1045189851 learned_addr learned my addr 172.21.15.191:0/1045189851 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:42.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4a950700 1 -- 172.21.15.191:0/1045189851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ef884d00 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4a950700 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x5641efcc5260 tx=0x5641efc928a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa3011b700 1 -- 172.21.15.191:0/1045189851 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641efcd6160 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa3011b700 1 -- 172.21.15.191:0/1045189851 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641efcd4180 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5641ef884340 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa3011b700 1 -- 172.21.15.191:0/1045189851 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5641efcd62c0 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.907+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641ef885d40 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.908+0000 7ffa3011b700 1 -- 172.21.15.191:0/1045189851 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5641f05ffb80 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.908+0000 7ffa3011b700 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efd4e000 0x5641efb47700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.908+0000 7ffa3011b700 1 -- 172.21.15.191:0/1045189851 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x5641ef8d7500 con 0x5641efcb1400 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.908+0000 7ffa4b952700 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efd4e000 0x5641efb47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.908+0000 7ffa4d1a7a80 5 note: GC not initialized 2024-04-27T04:35:42.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.908+0000 7ffa4b952700 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efd4e000 0x5641efb47700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5641efc51470 tx=0x5641efc925a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa2d916700 20 reqs_thread_entry: start 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa4d1a7a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa4d1a7a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa2b912700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:13 2024 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa2b0f0700 20 reqs_thread_entry: start 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa4d1a7a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.909+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb1800 con 0x5641ef85d800 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.910+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.910+0000 7ffa4d1a7a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.910+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efcb0800 con 0x5641efbeb000 2024-04-27T04:35:42.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.911+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.911+0000 7ffa3211f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.911+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94841195970560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4e400 con 0x5641ef85d800 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.912+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94841195970560] v33'209 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.912+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94841194396416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4e800 con 0x5641efbeb400 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.913+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94841194396416] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.913+0000 7ffa4d1a7a80 20 remove_watcher() i=0 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.913+0000 7ffa4d1a7a80 2 removed watcher, disabling cache 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.913+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94841194397568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4ec00 con 0x5641ef85d800 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.914+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94841194397568] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.914+0000 7ffa4d1a7a80 20 remove_watcher() i=1 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.914+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94841194398720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4f000 con 0x5641efbebc00 2024-04-27T04:35:42.946 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.915+0000 7ffa4b151700 1 -- 172.21.15.191:0/689875338 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94841194398720] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5200 con 0x5641efbebc00 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.916+0000 7ffa4d1a7a80 20 remove_watcher() i=2 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.916+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94841194399872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4f400 con 0x5641efbeb400 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.916+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94841194399872] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca5440 con 0x5641efbeb400 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.916+0000 7ffa4d1a7a80 20 remove_watcher() i=3 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.916+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94841194401024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4f800 con 0x5641ef85d800 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.917+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94841194401024] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641ef85d800 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.917+0000 7ffa4d1a7a80 20 remove_watcher() i=4 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.917+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94841194402176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4fc00 con 0x5641efbeb000 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.918+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94841194402176] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.918+0000 7ffa4d1a7a80 20 remove_watcher() i=5 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.918+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94841194403328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4e400 con 0x5641efbeb000 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.919+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94841194403328] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.919+0000 7ffa4d1a7a80 20 remove_watcher() i=6 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.919+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94841194404480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5641efd4ec00 con 0x5641efbeb000 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4b952700 1 -- 172.21.15.191:0/689875338 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94841194404480] v33'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5641efca46c0 con 0x5641efbeb000 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 20 remove_watcher() i=7 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5641efbeb000 msgr2=0x5641efb47180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x5641efbeb000 0x5641efb47180 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5641ef85d800 msgr2=0x5641efb45b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x5641ef85d800 0x5641efb45b80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.948 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641efbeb400 msgr2=0x5641efb46c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5641efbeb400 0x5641efb46c00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641efbebc00 msgr2=0x5641efb46100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5641efbebc00 0x5641efb46100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbeac00 msgr2=0x5641efb45600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbeac00 0x5641efb45600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5641efb3f890 tx=0x5641ef94d800 comp rx=0 tx=0).stop 2024-04-27T04:35:42.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 msgr2=0x5641efb44b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/689875338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85d400 0x5641efb44b00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5641f06ea420 tx=0x5641efba6d20 comp rx=0 tx=0).stop 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.920+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 reap_dead start 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4a950700 1 -- 172.21.15.191:0/689875338 reap_dead start 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 shutdown_connections 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 >> 172.21.15.191:0/689875338 conn(0x5641ef85cc00 msgr2=0x5641ef949800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 shutdown_connections 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/689875338 wait complete. 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efd4e000 msgr2=0x5641efb47700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efd4e000 0x5641efb47700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5641efc51470 tx=0x5641efc925a0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 msgr2=0x5641efb46680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x5641efcc5260 tx=0x5641efc928a0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 shutdown_connections 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efd4e000 0x5641efb47700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/1045189851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641efcb1400 0x5641efb46680 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 >> 172.21.15.191:0/1045189851 conn(0x5641efcb1000 msgr2=0x5641ef94a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 shutdown_connections 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/1045189851 wait complete. 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbea000 msgr2=0x5641efb45080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbea000 0x5641efb45080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5641f06eaf00 tx=0x5641efba6240 comp rx=0 tx=0).stop 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 msgr2=0x5641efb44580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5641f062de30 tx=0x5641ef94d920 comp rx=0 tx=0).stop 2024-04-27T04:35:42.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 shutdown_connections 2024-04-27T04:35:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5641efbea000 0x5641efb45080 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 --2- 172.21.15.191:0/2239699932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ef85c800 0x5641efb44580 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.921+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 >> 172.21.15.191:0/2239699932 conn(0x5641ef85c400 msgr2=0x5641ef949000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.922+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 shutdown_connections 2024-04-27T04:35:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:42.922+0000 7ffa4d1a7a80 1 -- 172.21.15.191:0/2239699932 wait complete. 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:42.953 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.954 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 5 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.955 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.956 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:42.957 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.958 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "create_bucket", 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 8, 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 8 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:42.960 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 8, 2024-04-27T04:35:42.961 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 8 2024-04-27T04:35:42.961 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.961 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:42.961 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:42.961 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:42.961 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}}]} 2024-04-27T04:35:42.961 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-27T04:35:42.961 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-27T04:35:42.962 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-04-27T04:35:43.009 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:43.009 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:43.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f926b089a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.037 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f926b089a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55741b145500 con 0x55741ae1ac00 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f926b089a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55741aebca00 con 0x55741ae1a800 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f9269033700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f9268832700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55741ae1ac00 0x55741af06800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52644/0 (socket says 172.21.15.191:52644) 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f9268832700 1 -- 172.21.15.191:0/831196246 learned_addr learned my addr 172.21.15.191:0/831196246 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f9269033700 1 -- 172.21.15.191:0/831196246 >> v1:172.21.15.138:6789/0 conn(0x55741ae1ac00 legacy=0x55741af06800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.007+0000 7f9269033700 1 -- 172.21.15.191:0/831196246 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae43d40 con 0x55741ae1a800 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f9269033700 1 --2- 172.21.15.191:0/831196246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55741b0fd9e0 tx=0x55741af0b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435247ad9077cc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f9268031700 1 -- 172.21.15.191:0/831196246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed34a0 con 0x55741ae1a800 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f9268031700 1 -- 172.21.15.191:0/831196246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741b145800 con 0x55741ae1a800 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f9268031700 1 -- 172.21.15.191:0/831196246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed3600 con 0x55741ae1a800 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 -- 172.21.15.191:0/831196246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 msgr2=0x55741b102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 --2- 172.21.15.191:0/831196246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55741b0fd9e0 tx=0x55741af0b920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.038 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 -- 172.21.15.191:0/831196246 shutdown_connections 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 --2- 172.21.15.191:0/831196246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 -- 172.21.15.191:0/831196246 >> 172.21.15.191:0/831196246 conn(0x55741ae1a400 msgr2=0x55741af06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 -- 172.21.15.191:0/831196246 shutdown_connections 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.008+0000 7f926b089a80 1 -- 172.21.15.191:0/831196246 wait complete. 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f926b089a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f926b089a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741aebc500 con 0x55741ae1a800 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f9269033700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f9269033700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58610/0 (socket says 172.21.15.191:58610) 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f9269033700 1 -- 172.21.15.191:0/3195682214 learned_addr learned my addr 172.21.15.191:0/3195682214 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.009+0000 7f9269033700 1 -- 172.21.15.191:0/3195682214 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae43ba0 con 0x55741ae1a800 2024-04-27T04:35:43.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f9269033700 1 --2- 172.21.15.191:0/3195682214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55741bbf3740 tx=0x55741af0b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b91017b8be0bac server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f9267830700 1 -- 172.21.15.191:0/3195682214 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed3760 con 0x55741ae1a800 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f9267830700 1 -- 172.21.15.191:0/3195682214 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741b145b00 con 0x55741ae1a800 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f9267830700 1 -- 172.21.15.191:0/3195682214 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed34a0 con 0x55741ae1a800 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 -- 172.21.15.191:0/3195682214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 msgr2=0x55741b102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 --2- 172.21.15.191:0/3195682214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55741bbf3740 tx=0x55741af0b800 comp rx=0 tx=0).stop 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 -- 172.21.15.191:0/3195682214 shutdown_connections 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 --2- 172.21.15.191:0/3195682214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 -- 172.21.15.191:0/3195682214 >> 172.21.15.191:0/3195682214 conn(0x55741ae1a400 msgr2=0x55741af07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 -- 172.21.15.191:0/3195682214 shutdown_connections 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 -- 172.21.15.191:0/3195682214 wait complete. 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.010+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926b089a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926b089a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741aebc3c0 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f9269033700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f9269033700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58622/0 (socket says 172.21.15.191:58622) 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f9269033700 1 -- 172.21.15.191:0/1306372922 learned_addr learned my addr 172.21.15.191:0/1306372922 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f9269033700 1 -- 172.21.15.191:0/1306372922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae43a00 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f9269033700 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55741bbf35f0 tx=0x55741af0b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926602d700 1 -- 172.21.15.191:0/1306372922 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed3600 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926602d700 1 -- 172.21.15.191:0/1306372922 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741b145c80 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55741ae43860 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926602d700 1 -- 172.21.15.191:0/1306372922 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed38c0 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.011+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741ae436c0 con 0x55741ae1a800 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926602d700 1 -- 172.21.15.191:0/1306372922 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55741bbbcb00 con 0x55741ae1a800 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926b089a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926602d700 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2000 0x55741b103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926b089a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741aebc000 con 0x55741ae1b400 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f926602d700 1 -- 172.21.15.191:0/1306372922 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55741ae95180 con 0x55741ae1a800 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f9269033700 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2000 0x55741b103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f9269834700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f9269834700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58626/0 (socket says 172.21.15.191:58626) 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.012+0000 7f9269834700 1 -- 172.21.15.191:0/120693952 learned_addr learned my addr 172.21.15.191:0/120693952 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f9269834700 1 -- 172.21.15.191:0/120693952 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae42000 con 0x55741ae1b400 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f9269033700 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2000 0x55741b103080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55741bca6ed0 tx=0x55741b168b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f9269834700 1 --2- 172.21.15.191:0/120693952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55741bca6210 tx=0x55741b168e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e78b24f72239092 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f9263828700 1 -- 172.21.15.191:0/120693952 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed3600 con 0x55741ae1b400 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f9263828700 1 -- 172.21.15.191:0/120693952 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741aebbc80 con 0x55741ae1b400 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f9263828700 1 -- 172.21.15.191:0/120693952 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed38c0 con 0x55741ae1b400 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 -- 172.21.15.191:0/120693952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 msgr2=0x55741b102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 --2- 172.21.15.191:0/120693952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55741bca6210 tx=0x55741b168e40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 -- 172.21.15.191:0/120693952 shutdown_connections 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 --2- 172.21.15.191:0/120693952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 -- 172.21.15.191:0/120693952 >> 172.21.15.191:0/120693952 conn(0x55741ae1ac00 msgr2=0x55741af07800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 -- 172.21.15.191:0/120693952 shutdown_connections 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.013+0000 7f926b089a80 1 -- 172.21.15.191:0/120693952 wait complete. 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f926b089a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f926b089a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741ae56640 con 0x55741ae1b400 2024-04-27T04:35:43.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9269834700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9269834700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58630/0 (socket says 172.21.15.191:58630) 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 learned_addr learned my addr 172.21.15.191:0/4245019470 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae42ea0 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9269834700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55741bca65d0 tx=0x55741b168a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9262025700 1 -- 172.21.15.191:0/4245019470 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed3760 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9262025700 1 -- 172.21.15.191:0/4245019470 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741aebbb00 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f9262025700 1 -- 172.21.15.191:0/4245019470 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741aed3600 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.014+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55741ae42820 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.015+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741ae429c0 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.015+0000 7f9262025700 1 -- 172.21.15.191:0/4245019470 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55741bbbcdc0 con 0x55741ae1b400 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.015+0000 7f9262025700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2c00 0x55741b103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.015+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.015+0000 7f9262025700 1 -- 172.21.15.191:0/4245019470 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55741ae95340 con 0x55741ae1b400 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f9268832700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2c00 0x55741b103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55741ae1b800 0x55741b103b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741ae1bc00 con 0x55741ae1b800 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f9269834700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55741ae1b800 0x55741b103b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f9268832700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2c00 0x55741b103600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55741b1a5cb0 tx=0x55741b2580c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f9269834700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55741ae1b800 0x55741b103b80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 20 realm 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3000 con 0x55741ae1b800 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.016+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3400 con 0x55741ae1b800 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3800 con 0x55741ae1b800 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55741b1a3c00 0x55741b104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3000 con 0x55741b1a3c00 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f9269033700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55741b1a3c00 0x55741b104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.017+0000 7f9269033700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55741b1a3c00 0x55741b104100 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f926b089a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f926b089a80 20 searching for the correct realm 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3400 con 0x55741b1a3c00 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f9269033700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55741b1a3400 0x55741b104c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3000 con 0x55741b1a3400 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f9268832700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55741b1a3400 0x55741b104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.018+0000 7f9268832700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55741b1a3400 0x55741b104c00 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9268832700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55741b1a3000 0x55741b105180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9269834700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55741b1a3000 0x55741b105180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9269834700 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55741b1a3000 0x55741b105180 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.019+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3400 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.020+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3400 2024-04-27T04:35:43.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.021+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.022+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.023+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3400 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3400 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.024+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3400 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3c00 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3400 2024-04-27T04:35:43.052 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.025+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741ae1bc00 con 0x55741ae1b800 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c400 con 0x55741ae1b800 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c800 con 0x55741b1a3c00 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 zone default found 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 4 Realm: () 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 zonegroup default 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.026+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26cc00 con 0x55741b1a3000 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d000 con 0x55741b1a3c00 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d400 con 0x55741b1a3c00 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 started sync module instance, tier type = 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.027+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d800 con 0x55741b1a3400 2024-04-27T04:35:43.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.028+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.028+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93957158574848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c800 con 0x55741b1a3400 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.028+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d000 con 0x55741ae1b800 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.029+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93957158574848] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.030+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.030+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93957158576000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d400 con 0x55741ae1b800 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.030+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c400 con 0x55741b1a3c00 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.031+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93957158576000] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.031+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.031+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93957158577152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26cc00 con 0x55741b1a3c00 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.031+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741ae1bc00 con 0x55741b1a3400 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.032+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.032+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93957158577152] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.032+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93957158578304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b1a3800 con 0x55741b1a3400 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.032+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741ae1b800 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.033+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.033+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93957158579456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d800 con 0x55741ae1b800 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.033+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c800 con 0x55741b1a3000 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.033+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93957158578304] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.034+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93957158579456] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.035+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.035+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93957158580608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26dc00 con 0x55741b1a3000 2024-04-27T04:35:43.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.035+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d000 con 0x55741b1a3000 2024-04-27T04:35:43.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.036+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93957158580608] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.036+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.036+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93957158581760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d400 con 0x55741b1a3000 2024-04-27T04:35:43.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.036+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c000 con 0x55741b1a3000 2024-04-27T04:35:43.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93957158581760] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93957158582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d800 con 0x55741b1a3000 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=0 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=1 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=2 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=3 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=4 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=5 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.037+0000 7f926b089a80 20 add_watcher() i=6 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.038+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93957158582912] v33'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.038+0000 7f926b089a80 20 add_watcher() i=7 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.038+0000 7f926b089a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.038+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c800 con 0x55741ae1b800 2024-04-27T04:35:43.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.038+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.038+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93957160140800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26dc00 con 0x55741ae1b800 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.039+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93957160140800] v33'210 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.039+0000 7f9250001700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.039+0000 7f926b089a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.039+0000 7f926b089a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.039+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.039+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f926b089a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f926b089a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741ae56780 con 0x55741b26d400 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f9268832700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f9268832700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58644/0 (socket says 172.21.15.191:58644) 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f9268832700 1 -- 172.21.15.191:0/2565102754 learned_addr learned my addr 172.21.15.191:0/2565102754 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f9268832700 1 -- 172.21.15.191:0/2565102754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae42b60 con 0x55741b26d400 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f9268832700 1 --2- 172.21.15.191:0/2565102754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55741b20f5f0 tx=0x55741b169f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a2a3043a70b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f924f800700 1 -- 172.21.15.191:0/2565102754 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741b28a000 con 0x55741b26d400 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f924f800700 1 -- 172.21.15.191:0/2565102754 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741b28e000 con 0x55741b26d400 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.040+0000 7f924f800700 1 -- 172.21.15.191:0/2565102754 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741b28a160 con 0x55741b26d400 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- 172.21.15.191:0/2565102754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 msgr2=0x55741b104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 --2- 172.21.15.191:0/2565102754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55741b20f5f0 tx=0x55741b169f80 comp rx=0 tx=0).stop 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- 172.21.15.191:0/2565102754 shutdown_connections 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 --2- 172.21.15.191:0/2565102754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- 172.21.15.191:0/2565102754 >> 172.21.15.191:0/2565102754 conn(0x55741b26d000 msgr2=0x55741af08000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- 172.21.15.191:0/2565102754 shutdown_connections 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- 172.21.15.191:0/2565102754 wait complete. 2024-04-27T04:35:43.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 Processor -- start 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- start start 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.041+0000 7f926b089a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741aebc000 con 0x55741b26d400 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f9268832700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f9268832700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58656/0 (socket says 172.21.15.191:58656) 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f9268832700 1 -- 172.21.15.191:0/1214310143 learned_addr learned my addr 172.21.15.191:0/1214310143 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f9268832700 1 -- 172.21.15.191:0/1214310143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741ae42d00 con 0x55741b26d400 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f9268832700 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55741b28c210 tx=0x55741b2584e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f924dffd700 1 -- 172.21.15.191:0/1214310143 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741b28a2c0 con 0x55741b26d400 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55741ae42340 con 0x55741b26d400 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f924dffd700 1 -- 172.21.15.191:0/1214310143 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55741b28e180 con 0x55741b26d400 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f924dffd700 1 -- 172.21.15.191:0/1214310143 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55741b28a000 con 0x55741b26d400 2024-04-27T04:35:43.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.042+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741ae42b60 con 0x55741b26d400 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.043+0000 7f924dffd700 1 -- 172.21.15.191:0/1214310143 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55741bbbdb80 con 0x55741b26d400 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.043+0000 7f924dffd700 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b310000 0x55741b105700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.043+0000 7f924dffd700 1 -- 172.21.15.191:0/1214310143 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55741ae95500 con 0x55741b26d400 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.043+0000 7f926b089a80 5 note: GC not initialized 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.043+0000 7f9269834700 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b310000 0x55741b105700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.044+0000 7f9269834700 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b310000 0x55741b105700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55741b20fec0 tx=0x55741b258360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.044+0000 7f924b7f8700 20 reqs_thread_entry: start 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.044+0000 7f926b089a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.044+0000 7f926b089a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.044+0000 7f92497f4700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:13 2024 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.044+0000 7f9248fd2700 20 reqs_thread_entry: start 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.045+0000 7f926b089a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.045+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26d800 con 0x55741ae1b800 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.045+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.045+0000 7f926b089a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.045+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b26c800 con 0x55741b1a3000 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.045+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.046+0000 7f9250001700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.046+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93957160140800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b310400 con 0x55741ae1b800 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.047+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93957160140800] v33'211 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.047+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93957158574848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b310800 con 0x55741b1a3400 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.048+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93957158574848] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.048+0000 7f926b089a80 20 remove_watcher() i=0 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.048+0000 7f926b089a80 2 removed watcher, disabling cache 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.048+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93957158576000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b310c00 con 0x55741ae1b800 2024-04-27T04:35:43.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.049+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93957158576000] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.049+0000 7f926b089a80 20 remove_watcher() i=1 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.049+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93957158577152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b311000 con 0x55741b1a3c00 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.050+0000 7f9269033700 1 -- 172.21.15.191:0/4245019470 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93957158577152] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251200 con 0x55741b1a3c00 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.050+0000 7f926b089a80 20 remove_watcher() i=2 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.050+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93957158578304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b311400 con 0x55741b1a3400 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.051+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93957158578304] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b251440 con 0x55741b1a3400 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.051+0000 7f926b089a80 20 remove_watcher() i=3 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.051+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93957158579456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b311800 con 0x55741ae1b800 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.052+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93957158579456] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741ae1b800 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.052+0000 7f926b089a80 20 remove_watcher() i=4 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.052+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93957158580608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b311c00 con 0x55741b1a3000 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.053+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93957158580608] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.053+0000 7f926b089a80 20 remove_watcher() i=5 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.053+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93957158581760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b310400 con 0x55741b1a3000 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.054+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93957158581760] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.054+0000 7f926b089a80 20 remove_watcher() i=6 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.054+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93957158582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55741b310c00 con 0x55741b1a3000 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.055+0000 7f9269834700 1 -- 172.21.15.191:0/4245019470 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93957158582912] v33'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55741b250b40 con 0x55741b1a3000 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.055+0000 7f926b089a80 20 remove_watcher() i=7 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.055+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55741b1a3000 msgr2=0x55741b105180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.055+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55741b1a3000 0x55741b105180 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.055+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55741ae1b800 msgr2=0x55741b103b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55741ae1b800 0x55741b103b80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55741b1a3400 msgr2=0x55741b104c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55741b1a3400 0x55741b104c00 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.076 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55741b1a3c00 msgr2=0x55741b104100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55741b1a3c00 0x55741b104100 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2c00 msgr2=0x55741b103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2c00 0x55741b103600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55741b1a5cb0 tx=0x55741b2580c0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 msgr2=0x55741b102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/4245019470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1b400 0x55741b102b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55741bca65d0 tx=0x55741b168a20 comp rx=0 tx=0).stop 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 reap_dead start 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f9268832700 1 -- 172.21.15.191:0/4245019470 reap_dead start 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 shutdown_connections 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 >> 172.21.15.191:0/4245019470 conn(0x55741ae1ac00 msgr2=0x55741af07800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 shutdown_connections 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/4245019470 wait complete. 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b310000 msgr2=0x55741b105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b310000 0x55741b105700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55741b20fec0 tx=0x55741b258360 comp rx=0 tx=0).stop 2024-04-27T04:35:43.077 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 msgr2=0x55741b104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55741b28c210 tx=0x55741b2584e0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 shutdown_connections 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b310000 0x55741b105700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 --2- 172.21.15.191:0/1214310143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741b26d400 0x55741b104680 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 >> 172.21.15.191:0/1214310143 conn(0x55741b26d000 msgr2=0x55741af08000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 shutdown_connections 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.056+0000 7f926b089a80 1 -- 172.21.15.191:0/1214310143 wait complete. 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2000 msgr2=0x55741b103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2000 0x55741b103080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55741bca6ed0 tx=0x55741b168b40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 msgr2=0x55741b102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55741bbf35f0 tx=0x55741af0b920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 shutdown_connections 2024-04-27T04:35:43.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55741b1a2000 0x55741b103080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 --2- 172.21.15.191:0/1306372922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55741ae1a800 0x55741b102580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 >> 172.21.15.191:0/1306372922 conn(0x55741ae1a400 msgr2=0x55741af07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 shutdown_connections 2024-04-27T04:35:43.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.057+0000 7f926b089a80 1 -- 172.21.15.191:0/1306372922 wait complete. 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.080 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_obj", 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:43.081 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:43.082 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.083 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "put_obj", 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.084 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 30000042, 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 11, 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 11 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.085 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:43.085 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}}]} 2024-04-27T04:35:43.085 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-27T04:35:43.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-27T04:35:43.086 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-04-27T04:35:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:43.133 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332d1d9a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332d1d9a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563ef7407500 con 0x563ef70dcc00 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332d1d9a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x563ef717ea00 con 0x563ef70dc800 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332b183700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332a982700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x563ef70dcc00 0x563ef71c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52646/0 (socket says 172.21.15.191:52646) 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332b183700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58670/0 (socket says 172.21.15.191:58670) 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332a982700 1 -- 172.21.15.191:0/3527923181 learned_addr learned my addr 172.21.15.191:0/3527923181 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332b183700 1 -- 172.21.15.191:0/3527923181 >> v1:172.21.15.138:6789/0 conn(0x563ef70dcc00 legacy=0x563ef71c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.131+0000 7f332b183700 1 -- 172.21.15.191:0/3527923181 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7105d40 con 0x563ef70dc800 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332b183700 1 --2- 172.21.15.191:0/3527923181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x563ef73bf9e0 tx=0x563ef71cd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29b223fa361bcf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332a181700 1 -- 172.21.15.191:0/3527923181 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef71954a0 con 0x563ef70dc800 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332a181700 1 -- 172.21.15.191:0/3527923181 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef7407800 con 0x563ef70dc800 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332a181700 1 -- 172.21.15.191:0/3527923181 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7195600 con 0x563ef70dc800 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3527923181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 msgr2=0x563ef73c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3527923181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x563ef73bf9e0 tx=0x563ef71cd920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3527923181 shutdown_connections 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3527923181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3527923181 >> 172.21.15.191:0/3527923181 conn(0x563ef70dc400 msgr2=0x563ef71c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3527923181 shutdown_connections 2024-04-27T04:35:43.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.132+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3527923181 wait complete. 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332d1d9a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332d1d9a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef717e500 con 0x563ef70dc800 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332b183700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332b183700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58678/0 (socket says 172.21.15.191:58678) 2024-04-27T04:35:43.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332b183700 1 -- 172.21.15.191:0/4172990709 learned_addr learned my addr 172.21.15.191:0/4172990709 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.133+0000 7f332b183700 1 -- 172.21.15.191:0/4172990709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7105ba0 con 0x563ef70dc800 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332b183700 1 --2- 172.21.15.191:0/4172990709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x563ef7eb5740 tx=0x563ef71cd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9875e434686bac server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f3329980700 1 -- 172.21.15.191:0/4172990709 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7195760 con 0x563ef70dc800 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f3329980700 1 -- 172.21.15.191:0/4172990709 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef7407b00 con 0x563ef70dc800 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f3329980700 1 -- 172.21.15.191:0/4172990709 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef71951e0 con 0x563ef70dc800 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4172990709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 msgr2=0x563ef73c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/4172990709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x563ef7eb5740 tx=0x563ef71cd800 comp rx=0 tx=0).stop 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4172990709 shutdown_connections 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/4172990709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4172990709 >> 172.21.15.191:0/4172990709 conn(0x563ef70dc400 msgr2=0x563ef71c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4172990709 shutdown_connections 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4172990709 wait complete. 2024-04-27T04:35:43.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.134+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332d1d9a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332d1d9a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef717e3c0 con 0x563ef70dc800 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332b183700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332b183700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58688/0 (socket says 172.21.15.191:58688) 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332b183700 1 -- 172.21.15.191:0/4083628896 learned_addr learned my addr 172.21.15.191:0/4083628896 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332b183700 1 -- 172.21.15.191:0/4083628896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7105a00 con 0x563ef70dc800 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332b183700 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x563ef7eb55f0 tx=0x563ef71cd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332817d700 1 -- 172.21.15.191:0/4083628896 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef71954a0 con 0x563ef70dc800 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332817d700 1 -- 172.21.15.191:0/4083628896 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef7407c80 con 0x563ef70dc800 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ef7105860 con 0x563ef70dc800 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332817d700 1 -- 172.21.15.191:0/4083628896 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7195600 con 0x563ef70dc800 2024-04-27T04:35:43.158 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.135+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ef71056c0 con 0x563ef70dc800 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332817d700 1 -- 172.21.15.191:0/4083628896 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x563ef7e7eb00 con 0x563ef70dc800 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332d1d9a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332d1d9a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef717e000 con 0x563ef70dd400 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332817d700 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466000 0x563ef73c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332817d700 1 -- 172.21.15.191:0/4083628896 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x563ef7157180 con 0x563ef70dc800 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332b183700 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466000 0x563ef73c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332b984700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332b984700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58694/0 (socket says 172.21.15.191:58694) 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.136+0000 7f332b984700 1 -- 172.21.15.191:0/3574715347 learned_addr learned my addr 172.21.15.191:0/3574715347 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332b984700 1 -- 172.21.15.191:0/3574715347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7104000 con 0x563ef70dd400 2024-04-27T04:35:43.159 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332b183700 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466000 0x563ef73c5080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x563ef7f54f90 tx=0x563ef742ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332b984700 1 --2- 172.21.15.191:0/3574715347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x563ef7f543c0 tx=0x563ef742ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e75735aa82f381 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f3325978700 1 -- 172.21.15.191:0/3574715347 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef71954a0 con 0x563ef70dd400 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f3325978700 1 -- 172.21.15.191:0/3574715347 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef717dc80 con 0x563ef70dd400 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f3325978700 1 -- 172.21.15.191:0/3574715347 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7195600 con 0x563ef70dd400 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3574715347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 msgr2=0x563ef73c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3574715347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x563ef7f543c0 tx=0x563ef742ae40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3574715347 shutdown_connections 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3574715347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3574715347 >> 172.21.15.191:0/3574715347 conn(0x563ef70dcc00 msgr2=0x563ef71c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3574715347 shutdown_connections 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.137+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3574715347 wait complete. 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.160 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332d1d9a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332d1d9a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef7118640 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332b984700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332b984700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58704/0 (socket says 172.21.15.191:58704) 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 learned_addr learned my addr 172.21.15.191:0/3131463101 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7104ea0 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.138+0000 7f332b984700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x563ef7f55f80 tx=0x563ef742aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.139+0000 7f3324175700 1 -- 172.21.15.191:0/3131463101 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7195760 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.139+0000 7f3324175700 1 -- 172.21.15.191:0/3131463101 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef717db00 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.139+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ef7104820 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.139+0000 7f3324175700 1 -- 172.21.15.191:0/3131463101 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef71954a0 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.139+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ef71049c0 con 0x563ef70dd400 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.139+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.161 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f3324175700 1 -- 172.21.15.191:0/3131463101 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x563ef7e7edc0 con 0x563ef70dd400 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f3324175700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466c00 0x563ef73c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f3324175700 1 -- 172.21.15.191:0/3131463101 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x563ef7157340 con 0x563ef70dd400 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332a982700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466c00 0x563ef73c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x563ef70dd800 0x563ef73c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef70ddc00 con 0x563ef70dd800 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332b984700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x563ef70dd800 0x563ef73c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332a982700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466c00 0x563ef73c5600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x563ef7477d70 tx=0x563ef75220c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332b984700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x563ef70dd800 0x563ef73c5b80 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332d1d9a80 20 realm 2024-04-27T04:35:43.162 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.140+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467000 con 0x563ef70dd800 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467400 con 0x563ef70dd800 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467800 con 0x563ef70dd800 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x563ef7467c00 0x563ef73c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467000 con 0x563ef7467c00 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332b183700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x563ef7467c00 0x563ef73c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.141+0000 7f332b183700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x563ef7467c00 0x563ef73c6100 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332d1d9a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332d1d9a80 20 searching for the correct realm 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467400 con 0x563ef7467c00 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332b183700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x563ef7467400 0x563ef73c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467000 con 0x563ef7467400 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332a982700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x563ef7467400 0x563ef73c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.142+0000 7f332a982700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x563ef7467400 0x563ef73c6c00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332a982700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x563ef7467000 0x563ef73c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332b984700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x563ef7467000 0x563ef73c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332b984700 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x563ef7467000 0x563ef73c7180 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.143+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467400 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.144+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.145+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467400 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.146+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.167 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.147+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.148+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.148+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.148+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.148+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467400 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.148+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.148+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467400 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.149+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467400 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467c00 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467400 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.150+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:43.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef70ddc00 con 0x563ef70dd800 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530400 con 0x563ef70dd800 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530800 con 0x563ef7467c00 2024-04-27T04:35:43.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 zone default found 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 4 Realm: () 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 zonegroup default 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.151+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530c00 con 0x563ef7467000 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531000 con 0x563ef7467c00 2024-04-27T04:35:43.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531400 con 0x563ef7467c00 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 started sync module instance, tier type = 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.152+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531800 con 0x563ef7467400 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.153+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.153+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94828435847936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530800 con 0x563ef7467400 2024-04-27T04:35:43.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.153+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531000 con 0x563ef70dd800 2024-04-27T04:35:43.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.154+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94828435847936] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.154+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.154+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94828435849088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531400 con 0x563ef70dd800 2024-04-27T04:35:43.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.154+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530400 con 0x563ef7467c00 2024-04-27T04:35:43.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.155+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.155+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94828435850240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530c00 con 0x563ef7467c00 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.155+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef70ddc00 con 0x563ef7467400 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.155+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94828435849088] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.156+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94828435850240] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.156+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.156+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94828435851392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7467800 con 0x563ef7467400 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.156+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef70dd800 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.157+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.157+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94828435851392] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.157+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94828435852544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531800 con 0x563ef70dd800 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.157+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530800 con 0x563ef7467000 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.158+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.158+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94828435852544] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.158+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94828435853696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531c00 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.158+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531000 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.159+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94828435853696] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.159+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94828435854848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531400 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530000 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94828435854848] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94828435856000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531800 con 0x563ef7467000 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=0 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=1 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=2 2024-04-27T04:35:43.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=3 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=4 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=5 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.160+0000 7f332d1d9a80 20 add_watcher() i=6 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.161+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94828435856000] v33'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.161+0000 7f332d1d9a80 20 add_watcher() i=7 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.161+0000 7f332d1d9a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.161+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530800 con 0x563ef70dd800 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.162+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.162+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94828437422080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531c00 con 0x563ef70dd800 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94828437422080] v33'212 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f3312151700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332d1d9a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:43.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332d1d9a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332d1d9a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.163+0000 7f332d1d9a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef7118780 con 0x563ef7531400 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f332a982700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f332a982700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58720/0 (socket says 172.21.15.191:58720) 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f332a982700 1 -- 172.21.15.191:0/2375432406 learned_addr learned my addr 172.21.15.191:0/2375432406 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f332a982700 1 -- 172.21.15.191:0/2375432406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7104b60 con 0x563ef7531400 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f332a982700 1 --2- 172.21.15.191:0/2375432406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x563ef7477470 tx=0x563ef742bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac3b332c145f762 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f3311950700 1 -- 172.21.15.191:0/2375432406 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7550000 con 0x563ef7531400 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f3311950700 1 -- 172.21.15.191:0/2375432406 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef755a000 con 0x563ef7531400 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.164+0000 7f3311950700 1 -- 172.21.15.191:0/2375432406 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7550160 con 0x563ef7531400 2024-04-27T04:35:43.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2375432406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 msgr2=0x563ef73c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/2375432406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x563ef7477470 tx=0x563ef742bf80 comp rx=0 tx=0).stop 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2375432406 shutdown_connections 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/2375432406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2375432406 >> 172.21.15.191:0/2375432406 conn(0x563ef7531000 msgr2=0x563ef71ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2375432406 shutdown_connections 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2375432406 wait complete. 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 Processor -- start 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- start start 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.165+0000 7f332d1d9a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef717e000 con 0x563ef7531400 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332a982700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332a982700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58734/0 (socket says 172.21.15.191:58734) 2024-04-27T04:35:43.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332a982700 1 -- 172.21.15.191:0/2049818345 learned_addr learned my addr 172.21.15.191:0/2049818345 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332a982700 1 -- 172.21.15.191:0/2049818345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef7104d00 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332a982700 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x563ef7552210 tx=0x563ef75224e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f331014d700 1 -- 172.21.15.191:0/2049818345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef75502c0 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f331014d700 1 -- 172.21.15.191:0/2049818345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ef755a180 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ef7104340 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f331014d700 1 -- 172.21.15.191:0/2049818345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563ef7550000 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.166+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ef7104b60 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.167+0000 7f331014d700 1 -- 172.21.15.191:0/2049818345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x563ef7e7fb80 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.167+0000 7f331014d700 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef75d2000 0x563ef73c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.167+0000 7f331014d700 1 -- 172.21.15.191:0/2049818345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x563ef7157500 con 0x563ef7531400 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.167+0000 7f332d1d9a80 5 note: GC not initialized 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.167+0000 7f332b984700 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef75d2000 0x563ef73c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.168+0000 7f332b984700 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef75d2000 0x563ef73c7700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x563ef74c8180 tx=0x563ef7522360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.168+0000 7f330d948700 20 reqs_thread_entry: start 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.168+0000 7f332d1d9a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.168+0000 7f332d1d9a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.168+0000 7f330b944700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:13 2024 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f330b122700 20 reqs_thread_entry: start 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f332d1d9a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7531800 con 0x563ef70dd800 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f332d1d9a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef7530800 con 0x563ef7467000 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.169+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.170+0000 7f3312151700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.170+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94828437422080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d2400 con 0x563ef70dd800 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.171+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94828437422080] v33'213 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.171+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94828435847936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d2800 con 0x563ef7467400 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.172+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94828435847936] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.172+0000 7f332d1d9a80 20 remove_watcher() i=0 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.172+0000 7f332d1d9a80 2 removed watcher, disabling cache 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.172+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94828435849088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d2c00 con 0x563ef70dd800 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.173+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94828435849088] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.173+0000 7f332d1d9a80 20 remove_watcher() i=1 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.173+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94828435850240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d3000 con 0x563ef7467c00 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.174+0000 7f332b183700 1 -- 172.21.15.191:0/3131463101 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94828435850240] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b200 con 0x563ef7467c00 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.174+0000 7f332d1d9a80 20 remove_watcher() i=2 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.174+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94828435851392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d3400 con 0x563ef7467400 2024-04-27T04:35:43.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.175+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94828435851392] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750b440 con 0x563ef7467400 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.175+0000 7f332d1d9a80 20 remove_watcher() i=3 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.175+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94828435852544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d3800 con 0x563ef70dd800 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.176+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94828435852544] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef70dd800 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.176+0000 7f332d1d9a80 20 remove_watcher() i=4 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.176+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94828435853696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d3c00 con 0x563ef7467000 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.177+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94828435853696] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.177+0000 7f332d1d9a80 20 remove_watcher() i=5 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.177+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94828435854848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d2400 con 0x563ef7467000 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.178+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94828435854848] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.178+0000 7f332d1d9a80 20 remove_watcher() i=6 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.178+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94828435856000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x563ef75d2c00 con 0x563ef7467000 2024-04-27T04:35:43.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332b984700 1 -- 172.21.15.191:0/3131463101 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94828435856000] v33'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563ef750ab40 con 0x563ef7467000 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 20 remove_watcher() i=7 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x563ef7467000 msgr2=0x563ef73c7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x563ef7467000 0x563ef73c7180 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x563ef70dd800 msgr2=0x563ef73c5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x563ef70dd800 0x563ef73c5b80 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x563ef7467400 msgr2=0x563ef73c6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x563ef7467400 0x563ef73c6c00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x563ef7467c00 msgr2=0x563ef73c6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x563ef7467c00 0x563ef73c6100 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466c00 msgr2=0x563ef73c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466c00 0x563ef73c5600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x563ef7477d70 tx=0x563ef75220c0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 msgr2=0x563ef73c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/3131463101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dd400 0x563ef73c4b00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x563ef7f55f80 tx=0x563ef742aa20 comp rx=0 tx=0).stop 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 reap_dead start 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 shutdown_connections 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332a982700 1 -- 172.21.15.191:0/3131463101 reap_dead start 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 >> 172.21.15.191:0/3131463101 conn(0x563ef70dcc00 msgr2=0x563ef71c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 shutdown_connections 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.179+0000 7f332d1d9a80 1 -- 172.21.15.191:0/3131463101 wait complete. 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef75d2000 msgr2=0x563ef73c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef75d2000 0x563ef73c7700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x563ef74c8180 tx=0x563ef7522360 comp rx=0 tx=0).stop 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 msgr2=0x563ef73c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x563ef7552210 tx=0x563ef75224e0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 shutdown_connections 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef75d2000 0x563ef73c7700 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/2049818345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef7531400 0x563ef73c6680 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 >> 172.21.15.191:0/2049818345 conn(0x563ef7531000 msgr2=0x563ef71ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 shutdown_connections 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/2049818345 wait complete. 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466000 msgr2=0x563ef73c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466000 0x563ef73c5080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x563ef7f54f90 tx=0x563ef742ab40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 msgr2=0x563ef73c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x563ef7eb55f0 tx=0x563ef71cd920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 shutdown_connections 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x563ef7466000 0x563ef73c5080 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 --2- 172.21.15.191:0/4083628896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ef70dc800 0x563ef73c4580 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 >> 172.21.15.191:0/4083628896 conn(0x563ef70dc400 msgr2=0x563ef71c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 shutdown_connections 2024-04-27T04:35:43.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.180+0000 7f332d1d9a80 1 -- 172.21.15.191:0/4083628896 wait complete. 2024-04-27T04:35:43.199 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:43.199 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:43.199 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.200 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:43.201 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.202 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:43.206 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.206 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.206 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.206 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.206 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_obj", 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:43.207 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 4, 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.208 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:43.208 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T04:35:43.209 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-27T04:35:43.209 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-27T04:35:43.209 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-04-27T04:35:43.256 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:43.256 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:43.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.254+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.254+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.254+0000 7f8215e1aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556b9f4bac00 0x556b9f7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.254+0000 7f8215e1aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556b9f7e5500 con 0x556b9f4ba800 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.254+0000 7f8215e1aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x556b9f55ca00 con 0x556b9f4bac00 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f82135c3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556b9f4bac00 0x556b9f7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f8213dc4700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x556b9f4ba800 0x556b9f5a6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52652/0 (socket says 172.21.15.191:52652) 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f8213dc4700 1 -- 172.21.15.191:0/2574775951 learned_addr learned my addr 172.21.15.191:0/2574775951 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f82135c3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556b9f4bac00 0x556b9f7a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58740/0 (socket says 172.21.15.191:58740) 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f82135c3700 1 -- 172.21.15.191:0/2574775951 >> v1:172.21.15.138:6789/0 conn(0x556b9f4ba800 legacy=0x556b9f5a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f82135c3700 1 -- 172.21.15.191:0/2574775951 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e3d40 con 0x556b9f4bac00 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f82135c3700 1 --2- 172.21.15.191:0/2574775951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bac00 0x556b9f7a2580 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x556b9f79d890 tx=0x556b9f5ab920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c7b290721d5c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f8212dc2700 1 -- 172.21.15.191:0/2574775951 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f5734a0 con 0x556b9f4bac00 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f8212dc2700 1 -- 172.21.15.191:0/2574775951 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f7e5800 con 0x556b9f4bac00 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.255+0000 7f8212dc2700 1 -- 172.21.15.191:0/2574775951 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f573600 con 0x556b9f4bac00 2024-04-27T04:35:43.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2574775951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bac00 msgr2=0x556b9f7a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2574775951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bac00 0x556b9f7a2580 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x556b9f79d890 tx=0x556b9f5ab920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2574775951 shutdown_connections 2024-04-27T04:35:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2574775951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bac00 0x556b9f7a2580 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2574775951 >> 172.21.15.191:0/2574775951 conn(0x556b9f4ba400 msgr2=0x556b9f5a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2574775951 shutdown_connections 2024-04-27T04:35:43.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2574775951 wait complete. 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.256+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8215e1aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8215e1aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9f55c500 con 0x556b9f4ba800 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8213dc4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8213dc4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58742/0 (socket says 172.21.15.191:58742) 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8213dc4700 1 -- 172.21.15.191:0/3522281476 learned_addr learned my addr 172.21.15.191:0/3522281476 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8213dc4700 1 -- 172.21.15.191:0/3522281476 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e3ba0 con 0x556b9f4ba800 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f8213dc4700 1 --2- 172.21.15.191:0/3522281476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x556b9f62b860 tx=0x556b9f5abb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4f729bf49ebfed server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f82125c1700 1 -- 172.21.15.191:0/3522281476 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f5731e0 con 0x556b9f4ba800 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f82125c1700 1 -- 172.21.15.191:0/3522281476 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f7e5b00 con 0x556b9f4ba800 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.257+0000 7f82125c1700 1 -- 172.21.15.191:0/3522281476 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f5734a0 con 0x556b9f4ba800 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3522281476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 msgr2=0x556b9f7a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3522281476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x556b9f62b860 tx=0x556b9f5abb60 comp rx=0 tx=0).stop 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3522281476 shutdown_connections 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3522281476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3522281476 >> 172.21.15.191:0/3522281476 conn(0x556b9f4ba400 msgr2=0x556b9f5a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3522281476 shutdown_connections 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3522281476 wait complete. 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8215e1aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9f55c3c0 con 0x556b9f4ba800 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8213dc4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8213dc4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58748/0 (socket says 172.21.15.191:58748) 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.258+0000 7f8213dc4700 1 -- 172.21.15.191:0/2626197807 learned_addr learned my addr 172.21.15.191:0/2626197807 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8213dc4700 1 -- 172.21.15.191:0/2626197807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e3a00 con 0x556b9f4ba800 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8213dc4700 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x556b9f62baa0 tx=0x556b9f5aba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8210dbe700 1 -- 172.21.15.191:0/2626197807 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f573600 con 0x556b9f4ba800 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8210dbe700 1 -- 172.21.15.191:0/2626197807 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f55bc80 con 0x556b9f4ba800 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b9f4e3860 con 0x556b9f4ba800 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8210dbe700 1 -- 172.21.15.191:0/2626197807 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f5731e0 con 0x556b9f4ba800 2024-04-27T04:35:43.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b9f4e36c0 con 0x556b9f4ba800 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8210dbe700 1 -- 172.21.15.191:0/2626197807 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556ba025cb00 con 0x556b9f4ba800 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.259+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f8215e1aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f8215e1aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9f55c000 con 0x556b9f4bb400 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f8210dbe700 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848000 0x556b9f7a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f8210dbe700 1 -- 172.21.15.191:0/2626197807 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556b9f535180 con 0x556b9f4ba800 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f8213dc4700 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848000 0x556b9f7a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f82145c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f82145c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58762/0 (socket says 172.21.15.191:58762) 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f82145c5700 1 -- 172.21.15.191:0/1277568189 learned_addr learned my addr 172.21.15.191:0/1277568189 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f82145c5700 1 -- 172.21.15.191:0/1277568189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e2000 con 0x556b9f4bb400 2024-04-27T04:35:43.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f8213dc4700 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848000 0x556b9f7a3080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x556ba032cae0 tx=0x556b9f804240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f82145c5700 1 --2- 172.21.15.191:0/1277568189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x556ba032c690 tx=0x556b9f8050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc268b3ebca2506 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f820e5b9700 1 -- 172.21.15.191:0/1277568189 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f573600 con 0x556b9f4bb400 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f820e5b9700 1 -- 172.21.15.191:0/1277568189 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f7e4300 con 0x556b9f4bb400 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.260+0000 7f820e5b9700 1 -- 172.21.15.191:0/1277568189 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f5731e0 con 0x556b9f4bb400 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- 172.21.15.191:0/1277568189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 msgr2=0x556b9f7a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/1277568189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x556ba032c690 tx=0x556b9f8050e0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- 172.21.15.191:0/1277568189 shutdown_connections 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/1277568189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- 172.21.15.191:0/1277568189 >> 172.21.15.191:0/1277568189 conn(0x556b9f4bac00 msgr2=0x556b9f5a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- 172.21.15.191:0/1277568189 shutdown_connections 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- 172.21.15.191:0/1277568189 wait complete. 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f8215e1aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9f4f6640 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f82145c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f82145c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58764/0 (socket says 172.21.15.191:58764) 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.261+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 learned_addr learned my addr 172.21.15.191:0/3262922925 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e2ea0 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f82145c5700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x556ba032df80 tx=0x556b9f804cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f820cdb6700 1 -- 172.21.15.191:0/3262922925 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f573a20 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f820cdb6700 1 -- 172.21.15.191:0/3262922925 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f55bc80 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f820cdb6700 1 -- 172.21.15.191:0/3262922925 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f573600 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b9f4e2820 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.262+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b9f4e29c0 con 0x556b9f4bb400 2024-04-27T04:35:43.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f820cdb6700 1 -- 172.21.15.191:0/3262922925 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556ba025cdc0 con 0x556b9f4bb400 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f820cdb6700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848c00 0x556b9f7a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f820cdb6700 1 -- 172.21.15.191:0/3262922925 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556b9f535340 con 0x556b9f4bb400 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f82135c3700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848c00 0x556b9f7a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556b9f4bb800 0x556b9f7a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f4bbc00 con 0x556b9f4bb800 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f82145c5700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556b9f4bb800 0x556b9f7a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.263+0000 7f82135c3700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848c00 0x556b9f7a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x556b9f79d890 tx=0x556b9f5ab800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f82145c5700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556b9f4bb800 0x556b9f7a3b80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 realm 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849000 con 0x556b9f4bb800 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849400 con 0x556b9f4bb800 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.264+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849800 con 0x556b9f4bb800 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556b9f849c00 0x556b9f7a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.288 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849000 con 0x556b9f849c00 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8213dc4700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556b9f849c00 0x556b9f7a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8213dc4700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556b9f849c00 0x556b9f7a4100 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 20 searching for the correct realm 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.265+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849400 con 0x556b9f849c00 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f8213dc4700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556b9f849400 0x556b9f7a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849000 con 0x556b9f849400 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f82135c3700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556b9f849400 0x556b9f7a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f82135c3700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556b9f849400 0x556b9f7a4c00 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.289 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f82135c3700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556b9f849000 0x556b9f7a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.266+0000 7f82145c5700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556b9f849000 0x556b9f7a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556b9f849000 0x556b9f7a5180 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.267+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.290 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849400 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.268+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849400 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.269+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.270+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849400 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.271+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849400 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.272+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849400 2024-04-27T04:35:43.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849c00 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.273+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849400 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f4bbc00 con 0x556b9f4bb800 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918400 con 0x556b9f4bb800 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918800 con 0x556b9f849c00 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 zone default found 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 4 Realm: () 2024-04-27T04:35:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 zonegroup default 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.274+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918c00 con 0x556b9f849000 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919000 con 0x556b9f849c00 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919400 con 0x556b9f849c00 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 started sync module instance, tier type = 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.275+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919800 con 0x556b9f849400 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.276+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.276+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93920725408512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918800 con 0x556b9f849400 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.276+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919000 con 0x556b9f4bb800 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.277+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93920725408512] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.277+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.277+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93920725409664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919400 con 0x556b9f4bb800 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.277+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918400 con 0x556b9f849c00 2024-04-27T04:35:43.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.278+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93920725409664] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.279+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.279+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93920725410816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918c00 con 0x556b9f849c00 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.279+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f4bbc00 con 0x556b9f849400 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.279+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93920725410816] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.280+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.280+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93920725411968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f849800 con 0x556b9f849400 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.280+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f4bb800 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.281+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.281+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93920725411968] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.281+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93920725413120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919800 con 0x556b9f4bb800 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.281+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918800 con 0x556b9f849000 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.282+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.282+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93920725414272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919c00 con 0x556b9f849000 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.282+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919000 con 0x556b9f849000 2024-04-27T04:35:43.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.282+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93920725413120] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.282+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93920725414272] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.283+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.283+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93920725415424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919400 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.283+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918000 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93920725415424] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93920725416576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919800 con 0x556b9f849000 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=0 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=1 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=2 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=3 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=4 2024-04-27T04:35:43.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=5 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.284+0000 7f8215e1aa80 20 add_watcher() i=6 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.285+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93920725416576] v33'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.285+0000 7f8215e1aa80 20 add_watcher() i=7 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.285+0000 7f8215e1aa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.285+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918800 con 0x556b9f4bb800 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.285+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.285+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93920727015424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919c00 con 0x556b9f4bb800 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.286+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93920727015424] v33'214 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.286+0000 7f81fad92700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f8215e1aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f8215e1aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.313 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f8215e1aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f8215e1aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9f4f6780 con 0x556b9f919400 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f82135c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f82135c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58774/0 (socket says 172.21.15.191:58774) 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f82135c3700 1 -- 172.21.15.191:0/2045478359 learned_addr learned my addr 172.21.15.191:0/2045478359 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f82135c3700 1 -- 172.21.15.191:0/2045478359 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e2b60 con 0x556b9f919400 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.287+0000 7f82135c3700 1 --2- 172.21.15.191:0/2045478359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x556b9f92e060 tx=0x556b9f5ab980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be5f3d13c4062d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f81fa591700 1 -- 172.21.15.191:0/2045478359 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f573760 con 0x556b9f919400 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f81fa591700 1 -- 172.21.15.191:0/2045478359 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f934000 con 0x556b9f919400 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f81fa591700 1 -- 172.21.15.191:0/2045478359 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f936000 con 0x556b9f919400 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2045478359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 msgr2=0x556b9f7a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2045478359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x556b9f92e060 tx=0x556b9f5ab980 comp rx=0 tx=0).stop 2024-04-27T04:35:43.314 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2045478359 shutdown_connections 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2045478359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2045478359 >> 172.21.15.191:0/2045478359 conn(0x556b9f919000 msgr2=0x556b9f5a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2045478359 shutdown_connections 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.288+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2045478359 wait complete. 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f8215e1aa80 1 Processor -- start 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f8215e1aa80 1 -- start start 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f8215e1aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f8215e1aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9f55c000 con 0x556b9f919400 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f82135c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f82135c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58780/0 (socket says 172.21.15.191:58780) 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f82135c3700 1 -- 172.21.15.191:0/3715569027 learned_addr learned my addr 172.21.15.191:0/3715569027 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f82135c3700 1 -- 172.21.15.191:0/3715569027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9f4e2d00 con 0x556b9f919400 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f82135c3700 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x556b9f92e2a0 tx=0x556b9f8f68a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f81f8d8e700 1 -- 172.21.15.191:0/3715569027 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f936160 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f81f8d8e700 1 -- 172.21.15.191:0/3715569027 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b9f934180 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.289+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b9f4e2340 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.290+0000 7f81f8d8e700 1 -- 172.21.15.191:0/3715569027 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556b9f9362c0 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.290+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b9f4e3d40 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.290+0000 7f81f8d8e700 1 -- 172.21.15.191:0/3715569027 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556ba025db80 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.290+0000 7f81f8d8e700 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f9b0000 0x556b9f7a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.290+0000 7f81f8d8e700 1 -- 172.21.15.191:0/3715569027 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556b9f535500 con 0x556b9f919400 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.291+0000 7f82145c5700 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f9b0000 0x556b9f7a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.291+0000 7f8215e1aa80 5 note: GC not initialized 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.291+0000 7f82145c5700 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f9b0000 0x556b9f7a5700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x556b9f84f650 tx=0x556b9f8f65a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.291+0000 7f81f6589700 20 reqs_thread_entry: start 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f8215e1aa80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:43.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f8215e1aa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f81f4585700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:13 2024 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f81f3d63700 20 reqs_thread_entry: start 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f8215e1aa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f919800 con 0x556b9f4bb800 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f8215e1aa80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.292+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f918800 con 0x556b9f849000 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.293+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv31 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.293+0000 7f81fad92700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.293+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93920727015424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b0400 con 0x556b9f4bb800 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.294+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93920727015424] v33'215 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.294+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93920725408512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b0800 con 0x556b9f849400 2024-04-27T04:35:43.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.295+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93920725408512] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.295+0000 7f8215e1aa80 20 remove_watcher() i=0 2024-04-27T04:35:43.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.295+0000 7f8215e1aa80 2 removed watcher, disabling cache 2024-04-27T04:35:43.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.295+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93920725409664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b0c00 con 0x556b9f4bb800 2024-04-27T04:35:43.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.296+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93920725409664] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.296+0000 7f8215e1aa80 20 remove_watcher() i=1 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.297+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93920725410816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b1000 con 0x556b9f849c00 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.297+0000 7f8213dc4700 1 -- 172.21.15.191:0/3262922925 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93920725410816] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5200 con 0x556b9f849c00 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.298+0000 7f8215e1aa80 20 remove_watcher() i=2 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.298+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93920725411968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b1400 con 0x556b9f849400 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.298+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93920725411968] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f5440 con 0x556b9f849400 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.298+0000 7f8215e1aa80 20 remove_watcher() i=3 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.298+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93920725413120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b1800 con 0x556b9f4bb800 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.299+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93920725413120] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f4bb800 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.299+0000 7f8215e1aa80 20 remove_watcher() i=4 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.299+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93920725414272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b1c00 con 0x556b9f849000 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.300+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93920725414272] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.301+0000 7f8215e1aa80 20 remove_watcher() i=5 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.301+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93920725415424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b0400 con 0x556b9f849000 2024-04-27T04:35:43.319 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.301+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93920725415424] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.301+0000 7f8215e1aa80 20 remove_watcher() i=6 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.301+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93920725416576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556b9f9b0c00 con 0x556b9f849000 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f82145c5700 1 -- 172.21.15.191:0/3262922925 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93920725416576] v33'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556b9f8f4b40 con 0x556b9f849000 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 20 remove_watcher() i=7 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556b9f849000 msgr2=0x556b9f7a5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556b9f849000 0x556b9f7a5180 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556b9f4bb800 msgr2=0x556b9f7a3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556b9f4bb800 0x556b9f7a3b80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556b9f849400 msgr2=0x556b9f7a4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x556b9f849400 0x556b9f7a4c00 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556b9f849c00 msgr2=0x556b9f7a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556b9f849c00 0x556b9f7a4100 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848c00 msgr2=0x556b9f7a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848c00 0x556b9f7a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x556b9f79d890 tx=0x556b9f5ab800 comp rx=0 tx=0).stop 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 msgr2=0x556b9f7a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3262922925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4bb400 0x556b9f7a2b00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x556ba032df80 tx=0x556b9f804cc0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.302+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 reap_dead start 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f82135c3700 1 -- 172.21.15.191:0/3262922925 reap_dead start 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 shutdown_connections 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 >> 172.21.15.191:0/3262922925 conn(0x556b9f4bac00 msgr2=0x556b9f5a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 shutdown_connections 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3262922925 wait complete. 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f9b0000 msgr2=0x556b9f7a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f9b0000 0x556b9f7a5700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x556b9f84f650 tx=0x556b9f8f65a0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 msgr2=0x556b9f7a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x556b9f92e2a0 tx=0x556b9f8f68a0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 shutdown_connections 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f9b0000 0x556b9f7a5700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/3715569027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f919400 0x556b9f7a4680 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 >> 172.21.15.191:0/3715569027 conn(0x556b9f919000 msgr2=0x556b9f5a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 shutdown_connections 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/3715569027 wait complete. 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848000 msgr2=0x556b9f7a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848000 0x556b9f7a3080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x556ba032cae0 tx=0x556b9f804240 comp rx=0 tx=0).stop 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 msgr2=0x556b9f7a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x556b9f62baa0 tx=0x556b9f5aba40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 shutdown_connections 2024-04-27T04:35:43.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556b9f848000 0x556b9f7a3080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 --2- 172.21.15.191:0/2626197807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b9f4ba800 0x556b9f7a2580 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.303+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 >> 172.21.15.191:0/2626197807 conn(0x556b9f4ba400 msgr2=0x556b9f5a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.304+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 shutdown_connections 2024-04-27T04:35:43.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.304+0000 7f8215e1aa80 1 -- 172.21.15.191:0/2626197807 wait complete. 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [ 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "buckets": [ 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "", 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.324 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [] 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo", 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 2, 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:43.325 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo2", 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.326 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo3", 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.327 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket": "myfoo4", 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "time": "2024-04-27T04:00:00.000000Z", 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "epoch": 1714190400, 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "owner": "foo", 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.328 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 0, 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 1, 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 1 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [ 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "foo", 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: "categories": [ 2024-04-27T04:35:43.329 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "category": "delete_bucket", 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "total": { 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_sent": 236, 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bytes_received": 0, 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "ops": 5, 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: "successful_ops": 4 2024-04-27T04:35:43.330 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.331 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.331 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:35:43.331 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:43.331 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}}]} 2024-04-27T04:35:43.331 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-27T04:35:43.331 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-27T04:35:43.331 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-04-27T04:35:43.384 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:43.384 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:43.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe503418a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe503418a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557ea1c8d500 con 0x557ea1962c00 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe503418a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x557ea1a04a00 con 0x557ea1962800 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe5013c2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe500bc1700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x557ea1962c00 0x557ea1a4e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52668/0 (socket says 172.21.15.191:52668) 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.382+0000 7fe500bc1700 1 -- 172.21.15.191:0/3118424860 learned_addr learned my addr 172.21.15.191:0/3118424860 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe5013c2700 1 -- 172.21.15.191:0/3118424860 >> v1:172.21.15.138:6789/0 conn(0x557ea1962c00 legacy=0x557ea1a4e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe5013c2700 1 -- 172.21.15.191:0/3118424860 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198bd40 con 0x557ea1962800 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe5013c2700 1 --2- 172.21.15.191:0/3118424860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x557ea1c459e0 tx=0x557ea1a53920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22c175b50b5fb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe5003c0700 1 -- 172.21.15.191:0/3118424860 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b4a0 con 0x557ea1962800 2024-04-27T04:35:43.388 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe5003c0700 1 -- 172.21.15.191:0/3118424860 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1c8d800 con 0x557ea1962800 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe5003c0700 1 -- 172.21.15.191:0/3118424860 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b600 con 0x557ea1962800 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe503418a80 1 -- 172.21.15.191:0/3118424860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 msgr2=0x557ea1c4a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe503418a80 1 --2- 172.21.15.191:0/3118424860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x557ea1c459e0 tx=0x557ea1a53920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe503418a80 1 -- 172.21.15.191:0/3118424860 shutdown_connections 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe503418a80 1 --2- 172.21.15.191:0/3118424860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.383+0000 7fe503418a80 1 -- 172.21.15.191:0/3118424860 >> 172.21.15.191:0/3118424860 conn(0x557ea1962400 msgr2=0x557ea1a4e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe503418a80 1 -- 172.21.15.191:0/3118424860 shutdown_connections 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe503418a80 1 -- 172.21.15.191:0/3118424860 wait complete. 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe503418a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe503418a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea1a04500 con 0x557ea1962800 2024-04-27T04:35:43.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe5013c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe5013c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58804/0 (socket says 172.21.15.191:58804) 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.384+0000 7fe5013c2700 1 -- 172.21.15.191:0/461541166 learned_addr learned my addr 172.21.15.191:0/461541166 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe5013c2700 1 -- 172.21.15.191:0/461541166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198bba0 con 0x557ea1962800 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe5013c2700 1 --2- 172.21.15.191:0/461541166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x557ea2729d40 tx=0x557ea1a53800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a256cd7d58885c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe4ffbbf700 1 -- 172.21.15.191:0/461541166 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b760 con 0x557ea1962800 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe4ffbbf700 1 -- 172.21.15.191:0/461541166 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1c8db00 con 0x557ea1962800 2024-04-27T04:35:43.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe4ffbbf700 1 -- 172.21.15.191:0/461541166 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b4a0 con 0x557ea1962800 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 -- 172.21.15.191:0/461541166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 msgr2=0x557ea1c4a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 --2- 172.21.15.191:0/461541166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x557ea2729d40 tx=0x557ea1a53800 comp rx=0 tx=0).stop 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 -- 172.21.15.191:0/461541166 shutdown_connections 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 --2- 172.21.15.191:0/461541166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 -- 172.21.15.191:0/461541166 >> 172.21.15.191:0/461541166 conn(0x557ea1962400 msgr2=0x557ea1a4f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 -- 172.21.15.191:0/461541166 shutdown_connections 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.385+0000 7fe503418a80 1 -- 172.21.15.191:0/461541166 wait complete. 2024-04-27T04:35:43.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe503418a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe503418a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea1a043c0 con 0x557ea1962800 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe5013c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe5013c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58818/0 (socket says 172.21.15.191:58818) 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe5013c2700 1 -- 172.21.15.191:0/1901914808 learned_addr learned my addr 172.21.15.191:0/1901914808 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe5013c2700 1 -- 172.21.15.191:0/1901914808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198ba00 con 0x557ea1962800 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.386+0000 7fe5013c2700 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x557ea2729620 tx=0x557ea1a53920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe4fe3bc700 1 -- 172.21.15.191:0/1901914808 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b600 con 0x557ea1962800 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe4fe3bc700 1 -- 172.21.15.191:0/1901914808 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1c8dc80 con 0x557ea1962800 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557ea198b860 con 0x557ea1962800 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe4fe3bc700 1 -- 172.21.15.191:0/1901914808 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b8c0 con 0x557ea1962800 2024-04-27T04:35:43.427 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ea198b6c0 con 0x557ea1962800 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe503418a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe503418a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea199e640 con 0x557ea1963400 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe4fe3bc700 1 -- 172.21.15.191:0/1901914808 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x557ea2704b00 con 0x557ea1962800 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.387+0000 7fe501bc3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe501bc3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58832/0 (socket says 172.21.15.191:58832) 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe4fe3bc700 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1cee000 0x557ea1c4b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe501bc3700 1 -- 172.21.15.191:0/2093122718 learned_addr learned my addr 172.21.15.191:0/2093122718 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe4fe3bc700 1 -- 172.21.15.191:0/1901914808 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x557ea19dd180 con 0x557ea1962800 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe5013c2700 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1cee000 0x557ea1c4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe501bc3700 1 -- 172.21.15.191:0/2093122718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198a000 con 0x557ea1963400 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe501bc3700 1 --2- 172.21.15.191:0/2093122718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x557ea1cf0060 tx=0x557ea1cb2b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3483986489d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.428 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe5013c2700 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1cee000 0x557ea1c4b080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x557ea27eef00 tx=0x557ea1cb2c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe4fbbb7700 1 -- 172.21.15.191:0/2093122718 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b600 con 0x557ea1963400 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe4fbbb7700 1 -- 172.21.15.191:0/2093122718 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1a03c80 con 0x557ea1963400 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe4fbbb7700 1 -- 172.21.15.191:0/2093122718 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b8c0 con 0x557ea1963400 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 -- 172.21.15.191:0/2093122718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 msgr2=0x557ea1c4ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 --2- 172.21.15.191:0/2093122718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x557ea1cf0060 tx=0x557ea1cb2b40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 -- 172.21.15.191:0/2093122718 shutdown_connections 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 --2- 172.21.15.191:0/2093122718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 -- 172.21.15.191:0/2093122718 >> 172.21.15.191:0/2093122718 conn(0x557ea1962c00 msgr2=0x557ea1a4f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 -- 172.21.15.191:0/2093122718 shutdown_connections 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.388+0000 7fe503418a80 1 -- 172.21.15.191:0/2093122718 wait complete. 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe503418a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe503418a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea199e8c0 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe501bc3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe501bc3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58840/0 (socket says 172.21.15.191:58840) 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 learned_addr learned my addr 172.21.15.191:0/3041857187 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.389+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198aea0 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x557ea27ee1b0 tx=0x557ea1cb25a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe4fa3b4700 1 -- 172.21.15.191:0/3041857187 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b760 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe4fa3b4700 1 -- 172.21.15.191:0/3041857187 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1a03b00 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557ea198a820 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe4fa3b4700 1 -- 172.21.15.191:0/3041857187 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1a1b600 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ea198a9c0 con 0x557ea1963400 2024-04-27T04:35:43.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe4fa3b4700 1 -- 172.21.15.191:0/3041857187 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x557ea2704dc0 con 0x557ea1963400 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.390+0000 7fe4fa3b4700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1ceec00 0x557ea1c4b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe4fa3b4700 1 -- 172.21.15.191:0/3041857187 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x557ea19dd340 con 0x557ea1963400 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1ceec00 0x557ea1c4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557ea1963800 0x557ea1c4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1963c00 con 0x557ea1963800 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557ea1963800 0x557ea1c4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557ea1963800 0x557ea1c4bb80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1ceec00 0x557ea1c4b600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x557ea1cf19e0 tx=0x557ea1da00c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 20 realm 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.391+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef000 con 0x557ea1963800 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef400 con 0x557ea1963800 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef800 con 0x557ea1963800 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.432 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557ea1cefc00 0x557ea1c4c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef000 con 0x557ea1cefc00 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe5013c2700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557ea1cefc00 0x557ea1c4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.392+0000 7fe5013c2700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557ea1cefc00 0x557ea1c4c100 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe503418a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe503418a80 20 searching for the correct realm 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef400 con 0x557ea1cefc00 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe5013c2700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557ea1cef400 0x557ea1c4cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef000 con 0x557ea1cef400 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557ea1cef400 0x557ea1c4cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.393+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557ea1cef400 0x557ea1c4cc00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.394+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.394+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557ea1cef000 0x557ea1c4d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.394+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.394+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557ea1cef000 0x557ea1c4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.394+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557ea1cef000 0x557ea1c4d180 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.395+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef400 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.396+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef400 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.397+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.398+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.398+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.398+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.398+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.398+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.398+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.399+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef400 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef400 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.400+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef400 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cefc00 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.401+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef400 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1963c00 con 0x557ea1963800 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4400 con 0x557ea1963800 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.402+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4800 con 0x557ea1cefc00 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 zone default found 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 4 Realm: () 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 zonegroup default 2024-04-27T04:35:43.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4c00 con 0x557ea1cef000 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5000 con 0x557ea1cefc00 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5400 con 0x557ea1cefc00 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 started sync module instance, tier type = 2024-04-27T04:35:43.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.403+0000 7fe503418a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.404+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5800 con 0x557ea1cef400 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.405+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.405+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94002368224000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4800 con 0x557ea1cef400 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.405+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5000 con 0x557ea1963800 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.406+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94002368224000] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.406+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.406+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94002368225152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5400 con 0x557ea1963800 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.406+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4400 con 0x557ea1cefc00 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.407+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94002368225152] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.407+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.407+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94002368226304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4c00 con 0x557ea1cefc00 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.407+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1963c00 con 0x557ea1cef400 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.408+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94002368226304] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.408+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.408+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94002368227456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1cef800 con 0x557ea1cef400 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.408+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1963800 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.409+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.409+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94002368228608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5800 con 0x557ea1963800 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.409+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4800 con 0x557ea1cef000 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.409+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94002368227456] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.410+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94002368228608] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.410+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.410+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94002368229760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5c00 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.410+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5000 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.411+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94002368229760] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.411+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.411+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94002368230912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5400 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.411+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4000 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94002368230912] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94002368232064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5800 con 0x557ea1cef000 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=0 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=1 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=2 2024-04-27T04:35:43.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=3 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=4 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=5 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.412+0000 7fe503418a80 20 add_watcher() i=6 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.413+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94002368232064] v33'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.413+0000 7fe503418a80 20 add_watcher() i=7 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.413+0000 7fe503418a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.413+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4800 con 0x557ea1963800 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.413+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.413+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94002369789952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5c00 con 0x557ea1963800 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.414+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94002369789952] v33'216 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.414+0000 7fe4e8390700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.414+0000 7fe503418a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.414+0000 7fe503418a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:43.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe503418a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 0x557ea1c4c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe503418a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea1a043c0 con 0x557ea1db5400 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe500bc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 0x557ea1c4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe500bc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 0x557ea1c4c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58850/0 (socket says 172.21.15.191:58850) 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe500bc1700 1 -- 172.21.15.191:0/3886021691 learned_addr learned my addr 172.21.15.191:0/3886021691 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe500bc1700 1 -- 172.21.15.191:0/3886021691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198ab60 con 0x557ea1db5400 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.415+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3886021691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 0x557ea1c4c680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x557ea1d57620 tx=0x557ea1cb3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b15a6fba8debee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe4e7b8f700 1 -- 172.21.15.191:0/3886021691 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1dd2000 con 0x557ea1db5400 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe4e7b8f700 1 -- 172.21.15.191:0/3886021691 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1dd6000 con 0x557ea1db5400 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe4e7b8f700 1 -- 172.21.15.191:0/3886021691 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1dd2160 con 0x557ea1db5400 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 -- 172.21.15.191:0/3886021691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 msgr2=0x557ea1c4c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 --2- 172.21.15.191:0/3886021691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 0x557ea1c4c680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x557ea1d57620 tx=0x557ea1cb3f80 comp rx=0 tx=0).stop 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 -- 172.21.15.191:0/3886021691 shutdown_connections 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 --2- 172.21.15.191:0/3886021691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db5400 0x557ea1c4c680 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 -- 172.21.15.191:0/3886021691 >> 172.21.15.191:0/3886021691 conn(0x557ea1db5000 msgr2=0x557ea1a50000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 -- 172.21.15.191:0/3886021691 shutdown_connections 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.416+0000 7fe503418a80 1 -- 172.21.15.191:0/3886021691 wait complete. 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe503418a80 1 Processor -- start 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe503418a80 1 -- start start 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe503418a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 0x557ea1c4d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe503418a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea1a04500 con 0x557ea1db4000 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe500bc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 0x557ea1c4d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe500bc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 0x557ea1c4d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58852/0 (socket says 172.21.15.191:58852) 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe500bc1700 1 -- 172.21.15.191:0/3951585253 learned_addr learned my addr 172.21.15.191:0/3951585253 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe500bc1700 1 -- 172.21.15.191:0/3951585253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea198ad00 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.417+0000 7fe500bc1700 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 0x557ea1c4d700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x557ea1dd4180 tx=0x557ea1da04e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe4e638c700 1 -- 172.21.15.191:0/3951585253 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1dd22c0 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe4e638c700 1 -- 172.21.15.191:0/3951585253 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ea1dd6180 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe4e638c700 1 -- 172.21.15.191:0/3951585253 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557ea1dd2000 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557ea198a340 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ea198ab60 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe4e638c700 1 -- 172.21.15.191:0/3951585253 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x557ea2705b80 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe4e638c700 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1e56000 0x557ea1e5c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.418+0000 7fe4e638c700 1 -- 172.21.15.191:0/3951585253 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x557ea19dd500 con 0x557ea1db4000 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.419+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1e56000 0x557ea1e5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.448 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.419+0000 7fe503418a80 5 note: GC not initialized 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.419+0000 7fe501bc3700 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1e56000 0x557ea1e5c000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x557ea1d57e90 tx=0x557ea1da0360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.419+0000 7fe4e3b87700 20 reqs_thread_entry: start 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe4e1b83700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:13 2024 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe4e1361700 20 reqs_thread_entry: start 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db4800 con 0x557ea1cef000 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e56400 con 0x557ea1963800 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.420+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e56800 con 0x557ea1cef000 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~346] v33'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.450 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe503418a80 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe503418a80 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe503418a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94002369792256 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e56c00 con 0x557ea1cef000 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== watch-notify(notify (1) cookie 94002368232064 notify 141733920791 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x557ea1d55d40 con 0x557ea1cef000 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe4fbbb7700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920791 cookie 94002368232064 notifier 4788 bl.length()=522 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe4fbbb7700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe4fbbb7700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(65 notify.7 [notify cookie 94002369792256 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe4fbbb7700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1eca000 con 0x557ea1cef000 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== watch-notify(notify_complete (2) cookie 94002369792256 notify 141733920791 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x557ea1e21d40 con 0x557ea1cef000 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.422+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e57000 con 0x557ea1963800 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.423+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v33'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.451 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.423+0000 7fe503418a80 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T04:35:43.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.423+0000 7fe503418a80 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T04:35:43.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.423+0000 7fe503418a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T04:35:43.452 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.423+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94002369792256 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e57400 con 0x557ea1cef000 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== watch-notify(notify (1) cookie 94002368229760 notify 141733920792 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x557ea1e21d40 con 0x557ea1cef000 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe4fb3b6700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920792 cookie 94002368229760 notifier 4788 bl.length()=177 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe4fb3b6700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe4fb3b6700 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(68 notify.5 [notify cookie 94002369792256 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe4fb3b6700 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1db5c00 con 0x557ea1cef000 2024-04-27T04:35:43.453 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.424+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== watch-notify(notify_complete (2) cookie 94002369792256 notify 141733920792 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x557ea1d55a00 con 0x557ea1cef000 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "bar", 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Bar", 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "", 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:35:43.454 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "bar", 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:43.455 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:35:43.456 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:35:43.457 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:43.457 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:35:43.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.425+0000 7fe4e8390700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.425+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94002369789952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e57800 con 0x557ea1963800 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.426+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94002369789952] v33'217 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.426+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94002368224000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1e57c00 con 0x557ea1cef400 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.427+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94002368224000] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.427+0000 7fe503418a80 20 remove_watcher() i=0 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.427+0000 7fe503418a80 2 removed watcher, disabling cache 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.427+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94002368225152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1eca000 con 0x557ea1963800 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.428+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94002368225152] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.428+0000 7fe503418a80 20 remove_watcher() i=1 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.428+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94002368226304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1eca800 con 0x557ea1cefc00 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.429+0000 7fe5013c2700 1 -- 172.21.15.191:0/3041857187 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94002368226304] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1200 con 0x557ea1cefc00 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.429+0000 7fe503418a80 20 remove_watcher() i=2 2024-04-27T04:35:43.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.429+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94002368227456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1ecac00 con 0x557ea1cef400 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.430+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94002368227456] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db1440 con 0x557ea1cef400 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.430+0000 7fe503418a80 20 remove_watcher() i=3 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.430+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94002368228608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1ecb000 con 0x557ea1963800 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.431+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94002368228608] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1963800 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.431+0000 7fe503418a80 20 remove_watcher() i=4 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.431+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94002368229760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1ecb400 con 0x557ea1cef000 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.432+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94002368229760] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.432+0000 7fe503418a80 20 remove_watcher() i=5 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.432+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94002368230912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1ecb800 con 0x557ea1cef000 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.433+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94002368230912] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.433+0000 7fe503418a80 20 remove_watcher() i=6 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.433+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94002368232064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557ea1ecbc00 con 0x557ea1cef000 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe501bc3700 1 -- 172.21.15.191:0/3041857187 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94002368232064] v33'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557ea1db0b40 con 0x557ea1cef000 2024-04-27T04:35:43.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 20 remove_watcher() i=7 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557ea1cef000 msgr2=0x557ea1c4d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x557ea1cef000 0x557ea1c4d180 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557ea1963800 msgr2=0x557ea1c4bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x557ea1963800 0x557ea1c4bb80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557ea1cef400 msgr2=0x557ea1c4cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x557ea1cef400 0x557ea1c4cc00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557ea1cefc00 msgr2=0x557ea1c4c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x557ea1cefc00 0x557ea1c4c100 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1ceec00 msgr2=0x557ea1c4b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1ceec00 0x557ea1c4b600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x557ea1cf19e0 tx=0x557ea1da00c0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 msgr2=0x557ea1c4ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 --2- 172.21.15.191:0/3041857187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1963400 0x557ea1c4ab00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x557ea27ee1b0 tx=0x557ea1cb25a0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 reap_dead start 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe500bc1700 1 -- 172.21.15.191:0/3041857187 reap_dead start 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 shutdown_connections 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 >> 172.21.15.191:0/3041857187 conn(0x557ea1962c00 msgr2=0x557ea1a4f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 shutdown_connections 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.434+0000 7fe503418a80 1 -- 172.21.15.191:0/3041857187 wait complete. 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1e56000 msgr2=0x557ea1e5c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1e56000 0x557ea1e5c000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x557ea1d57e90 tx=0x557ea1da0360 comp rx=0 tx=0).stop 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 msgr2=0x557ea1c4d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 0x557ea1c4d700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x557ea1dd4180 tx=0x557ea1da04e0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 shutdown_connections 2024-04-27T04:35:43.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1e56000 0x557ea1e5c000 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/3951585253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1db4000 0x557ea1c4d700 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 >> 172.21.15.191:0/3951585253 conn(0x557ea1db5000 msgr2=0x557ea1a50000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 shutdown_connections 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/3951585253 wait complete. 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1cee000 msgr2=0x557ea1c4b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1cee000 0x557ea1c4b080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x557ea27eef00 tx=0x557ea1cb2c60 comp rx=0 tx=0).stop 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 msgr2=0x557ea1c4a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x557ea2729620 tx=0x557ea1a53920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 shutdown_connections 2024-04-27T04:35:43.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x557ea1cee000 0x557ea1c4b080 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 --2- 172.21.15.191:0/1901914808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ea1962800 0x557ea1c4a580 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 >> 172.21.15.191:0/1901914808 conn(0x557ea1962400 msgr2=0x557ea1a4f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 shutdown_connections 2024-04-27T04:35:43.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.435+0000 7fe503418a80 1 -- 172.21.15.191:0/1901914808 wait complete. 2024-04-27T04:35:43.468 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:35:43.481 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:43.486 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-04-27T04:35:43.487 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-27T04:35:43.487 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-27T04:35:43.487 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-04-27T04:35:43.534 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:43.534 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d96eb1a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a7833c4c00 0x55a7836ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d96eb1a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a7836ef500 con 0x55a7833c4800 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d96eb1a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a783466a00 con 0x55a7833c4c00 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d9465a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55a7833c4c00 0x55a7836ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d94e5b700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55a7833c4800 0x55a7834b0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:52680/0 (socket says 172.21.15.191:52680) 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.532+0000 7f0d94e5b700 1 -- 172.21.15.191:0/218420813 learned_addr learned my addr 172.21.15.191:0/218420813 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:43.536 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d9465a700 1 -- 172.21.15.191:0/218420813 >> v1:172.21.15.138:6789/0 conn(0x55a7833c4800 legacy=0x55a7834b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d9465a700 1 -- 172.21.15.191:0/218420813 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833edd40 con 0x55a7833c4c00 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d9465a700 1 --2- 172.21.15.191:0/218420813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4c00 0x55a7836ac580 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x55a7836a7890 tx=0x55a7834b5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1c58194d4a568f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d93e59700 1 -- 172.21.15.191:0/218420813 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d4a0 con 0x55a7833c4c00 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d93e59700 1 -- 172.21.15.191:0/218420813 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7836ef800 con 0x55a7833c4c00 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d93e59700 1 -- 172.21.15.191:0/218420813 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d600 con 0x55a7833c4c00 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/218420813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4c00 msgr2=0x55a7836ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/218420813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4c00 0x55a7836ac580 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x55a7836a7890 tx=0x55a7834b5920 comp rx=0 tx=0).stop 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/218420813 shutdown_connections 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/218420813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4c00 0x55a7836ac580 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/218420813 >> 172.21.15.191:0/218420813 conn(0x55a7833c4400 msgr2=0x55a7834b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/218420813 shutdown_connections 2024-04-27T04:35:43.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.533+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/218420813 wait complete. 2024-04-27T04:35:43.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.534+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.534+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.534+0000 7f0d96eb1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.534+0000 7f0d96eb1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a783466500 con 0x55a7833c4800 2024-04-27T04:35:43.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.534+0000 7f0d94e5b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d94e5b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58876/0 (socket says 172.21.15.191:58876) 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d94e5b700 1 -- 172.21.15.191:0/4204981108 learned_addr learned my addr 172.21.15.191:0/4204981108 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d94e5b700 1 -- 172.21.15.191:0/4204981108 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833edba0 con 0x55a7833c4800 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/4204981108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55a78419d890 tx=0x55a7834b5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34069bda6baa9553 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d93658700 1 -- 172.21.15.191:0/4204981108 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d1e0 con 0x55a7833c4800 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d93658700 1 -- 172.21.15.191:0/4204981108 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7836efb00 con 0x55a7833c4800 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d93658700 1 -- 172.21.15.191:0/4204981108 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d4a0 con 0x55a7833c4800 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/4204981108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 msgr2=0x55a7836ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/4204981108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55a78419d890 tx=0x55a7834b5b60 comp rx=0 tx=0).stop 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/4204981108 shutdown_connections 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/4204981108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/4204981108 >> 172.21.15.191:0/4204981108 conn(0x55a7833c4400 msgr2=0x55a7834b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/4204981108 shutdown_connections 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.535+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/4204981108 wait complete. 2024-04-27T04:35:43.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d96eb1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d96eb1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7834663c0 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d94e5b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d94e5b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58880/0 (socket says 172.21.15.191:58880) 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d94e5b700 1 -- 172.21.15.191:0/3328925473 learned_addr learned my addr 172.21.15.191:0/3328925473 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d94e5b700 1 -- 172.21.15.191:0/3328925473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833eda00 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.536+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55a78419da70 tx=0x55a7834b5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d91e55700 1 -- 172.21.15.191:0/3328925473 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d600 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d91e55700 1 -- 172.21.15.191:0/3328925473 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a783465c80 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7833ed860 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d91e55700 1 -- 172.21.15.191:0/3328925473 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d1e0 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7833ed6c0 con 0x55a7833c4800 2024-04-27T04:35:43.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d91e55700 1 -- 172.21.15.191:0/3328925473 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a784166b00 con 0x55a7833c4800 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d96eb1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d96eb1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a783400640 con 0x55a7833c5400 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d91e55700 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754000 0x55a7836ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d91e55700 1 -- 172.21.15.191:0/3328925473 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55a78343f180 con 0x55a7833c4800 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.537+0000 7f0d9565c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d9565c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58886/0 (socket says 172.21.15.191:58886) 2024-04-27T04:35:43.562 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d9565c700 1 -- 172.21.15.191:0/3586597845 learned_addr learned my addr 172.21.15.191:0/3586597845 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754000 0x55a7836ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d9565c700 1 -- 172.21.15.191:0/3586597845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833ec000 con 0x55a7833c5400 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754000 0x55a7836ad080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55a784242ab0 tx=0x55a783710240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d9565c700 1 --2- 172.21.15.191:0/3586597845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55a7842426c0 tx=0x55a7837110e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d01eb1cdde039fa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d8f650700 1 -- 172.21.15.191:0/3586597845 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d600 con 0x55a7833c5400 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d8f650700 1 -- 172.21.15.191:0/3586597845 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7836ee300 con 0x55a7833c5400 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.538+0000 7f0d8f650700 1 -- 172.21.15.191:0/3586597845 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d1e0 con 0x55a7833c5400 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3586597845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 msgr2=0x55a7836acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/3586597845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55a7842426c0 tx=0x55a7837110e0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3586597845 shutdown_connections 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/3586597845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3586597845 >> 172.21.15.191:0/3586597845 conn(0x55a7833c4c00 msgr2=0x55a7834b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3586597845 shutdown_connections 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3586597845 wait complete. 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d96eb1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7834008c0 con 0x55a7833c5400 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d9565c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d9565c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58898/0 (socket says 172.21.15.191:58898) 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.539+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 learned_addr learned my addr 172.21.15.191:0/2122617681 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833ecea0 con 0x55a7833c5400 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d9565c700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55a7842424b0 tx=0x55a783710d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d8de4d700 1 -- 172.21.15.191:0/2122617681 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347da20 con 0x55a7833c5400 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d8de4d700 1 -- 172.21.15.191:0/2122617681 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a783465c80 con 0x55a7833c5400 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7833ec820 con 0x55a7833c5400 2024-04-27T04:35:43.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d8de4d700 1 -- 172.21.15.191:0/2122617681 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d600 con 0x55a7833c5400 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.540+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7833ec9c0 con 0x55a7833c5400 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d8de4d700 1 -- 172.21.15.191:0/2122617681 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a784166dc0 con 0x55a7833c5400 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d8de4d700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754c00 0x55a7836ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d8de4d700 1 -- 172.21.15.191:0/2122617681 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55a78343f340 con 0x55a7833c5400 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d9465a700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754c00 0x55a7836ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a7833c5800 0x55a7836adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7833c5c00 con 0x55a7833c5800 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d9565c700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a7833c5800 0x55a7836adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.541+0000 7f0d9465a700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754c00 0x55a7836ad600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55a7836a7890 tx=0x55a7834b5800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d9565c700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a7833c5800 0x55a7836adb80 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 realm 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755000 con 0x55a7833c5800 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755400 con 0x55a7833c5800 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755800 con 0x55a7833c5800 2024-04-27T04:35:43.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a783755c00 0x55a7836ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.542+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755000 con 0x55a783755c00 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.543+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a783755c00 0x55a7836ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.543+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a783755c00 0x55a7836ae100 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.543+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.543+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.543+0000 7f0d96eb1a80 20 searching for the correct realm 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.543+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755400 con 0x55a783755c00 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d94e5b700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a783755400 0x55a7836aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755000 con 0x55a783755400 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d9465a700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a783755400 0x55a7836aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d9465a700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a783755400 0x55a7836aec00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d9465a700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a783755000 0x55a7836af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.544+0000 7f0d9565c700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a783755000 0x55a7836af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a783755000 0x55a7836af180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.545+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755400 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.546+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755400 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.547+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.548+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.549+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755400 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.571 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755400 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.550+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755400 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755c00 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.551+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755400 2024-04-27T04:35:43.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7833c5c00 con 0x55a7833c5800 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c400 con 0x55a7833c5800 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:43.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a783755c00 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 zone default found 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 4 Realm: () 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 zonegroup default 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.552+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381cc00 con 0x55a783755000 2024-04-27T04:35:43.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d000 con 0x55a783755c00 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d400 con 0x55a783755c00 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 started sync module instance, tier type = 2024-04-27T04:35:43.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.553+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d800 con 0x55a783755400 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.554+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.554+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94177952676608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a783755400 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.554+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d000 con 0x55a7833c5800 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.555+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94177952676608] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.555+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.555+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94177952677760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d400 con 0x55a7833c5800 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.555+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c400 con 0x55a783755c00 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.556+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94177952677760] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.556+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.556+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94177952678912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381cc00 con 0x55a783755c00 2024-04-27T04:35:43.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.557+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7833c5c00 con 0x55a783755400 2024-04-27T04:35:43.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.557+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94177952678912] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.558+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.558+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94177952680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783755800 con 0x55a783755400 2024-04-27T04:35:43.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.558+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a7833c5800 2024-04-27T04:35:43.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.558+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.558+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94177952681216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d800 con 0x55a7833c5800 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.558+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.559+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94177952680064] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.559+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.559+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94177952682368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381dc00 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.559+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94177952681216] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.559+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d000 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.560+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(56 notify.6 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.560+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94177952683520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d400 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.560+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c000 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94177952682368] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94177952683520] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94177952684672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d800 con 0x55a783755000 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=0 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=1 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=2 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=3 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=4 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=5 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.561+0000 7f0d96eb1a80 20 add_watcher() i=6 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.562+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94177952684672] v33'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.562+0000 7f0d96eb1a80 20 add_watcher() i=7 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.562+0000 7f0d96eb1a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.562+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a7833c5800 2024-04-27T04:35:43.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.563+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.563+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94177954275328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381dc00 con 0x55a7833c5800 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94177954275328] v33'218 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d7be29700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d96eb1a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d96eb1a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d96eb1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d96eb1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7834663c0 con 0x55a78381d400 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.564+0000 7f0d9465a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d9465a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58900/0 (socket says 172.21.15.191:58900) 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d9465a700 1 -- 172.21.15.191:0/2513425058 learned_addr learned my addr 172.21.15.191:0/2513425058 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.601 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d9465a700 1 -- 172.21.15.191:0/2513425058 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833ecb60 con 0x55a78381d400 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d9465a700 1 --2- 172.21.15.191:0/2513425058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55a7837ab830 tx=0x55a7834b5980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4acd64d2ba21fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d7b628700 1 -- 172.21.15.191:0/2513425058 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a78347d760 con 0x55a78381d400 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d7b628700 1 -- 172.21.15.191:0/2513425058 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a78383e000 con 0x55a78381d400 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.565+0000 7f0d7b628700 1 -- 172.21.15.191:0/2513425058 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a783840000 con 0x55a78381d400 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2513425058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 msgr2=0x55a7836ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2513425058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55a7837ab830 tx=0x55a7834b5980 comp rx=0 tx=0).stop 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2513425058 shutdown_connections 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2513425058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2513425058 >> 172.21.15.191:0/2513425058 conn(0x55a78381d000 msgr2=0x55a7834b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2513425058 shutdown_connections 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2513425058 wait complete. 2024-04-27T04:35:43.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 Processor -- start 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- start start 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d96eb1a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a783466500 con 0x55a78381d400 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d9465a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d9465a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:58912/0 (socket says 172.21.15.191:58912) 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.566+0000 7f0d9465a700 1 -- 172.21.15.191:0/1133112713 learned_addr learned my addr 172.21.15.191:0/1133112713 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d9465a700 1 -- 172.21.15.191:0/1133112713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7833ecd00 con 0x55a78381d400 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d9465a700 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55a783832210 tx=0x55a7838048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d79e25700 1 -- 172.21.15.191:0/1133112713 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a783840160 con 0x55a78381d400 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d79e25700 1 -- 172.21.15.191:0/1133112713 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a78383e180 con 0x55a78381d400 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d79e25700 1 -- 172.21.15.191:0/1133112713 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a7838402c0 con 0x55a78381d400 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7833ec340 con 0x55a78381d400 2024-04-27T04:35:43.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.567+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7833edd40 con 0x55a78381d400 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.568+0000 7f0d79e25700 1 -- 172.21.15.191:0/1133112713 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a784167b80 con 0x55a78381d400 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.568+0000 7f0d79e25700 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a7838b6000 0x55a7836af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.568+0000 7f0d79e25700 1 -- 172.21.15.191:0/1133112713 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55a78343f500 con 0x55a78381d400 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.568+0000 7f0d96eb1a80 5 note: GC not initialized 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.568+0000 7f0d9565c700 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a7838b6000 0x55a7836af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.568+0000 7f0d9565c700 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a7838b6000 0x55a7836af700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a7837abd70 tx=0x55a7838045a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d77620700 20 reqs_thread_entry: start 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d96eb1a80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d96eb1a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d7561c700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:14 2024 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:43.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d74dfa700 20 reqs_thread_entry: start 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d96eb1a80 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.569+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d800 con 0x55a783755000 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.570+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(62 bar [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.570+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.570+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.570+0000 7f0d96eb1a80 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.570+0000 7f0d96eb1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.570+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a783755400 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.571+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(63 bud [create,call,writefull 0~181] v33'1 uv1 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.571+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.571+0000 7f0d96eb1a80 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:35:43.605 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.571+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.571+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 94177954277632 in=369b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6400 con 0x55a783755000 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== watch-notify(notify (1) cookie 94177952682368 notify 141733920794 ret 0) v3 ==== 399+0+0 (crc 0 0 0) 0x55a7837c1d40 con 0x55a783755000 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d8f650700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920794 cookie 94177952682368 notifier 4796 bl.length()=357 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d8f650700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d8f650700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 notify.5 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d8f650700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928000 con 0x55a783755000 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920794 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387fba0 con 0x55a783755000 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6800 con 0x55a7833c5800 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d96eb1a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 : miss 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.572+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 8.9 8:91bc09d5:root::.bucket.meta.myfoo6%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6c00 con 0x55a783755400 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 .bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv44 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=353 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x17 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.573+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 8.9 8:91bc09d5:root::.bucket.meta.myfoo6%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7400 con 0x55a783755400 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 .bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v33'45 uv45 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x17 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d96eb1a80 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d96eb1a80 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7, ofs0, ns] 2024-04-27T04:35:43.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94177954277632 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7c00 con 0x55a783755000 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== watch-notify(notify (1) cookie 94177952683520 notify 141733920795 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x55a78387fba0 con 0x55a783755000 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d8f650700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920795 cookie 94177952683520 notifier 4796 bl.length()=721 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d8f650700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x17 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(69 notify.6 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d8f650700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d8f650700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.574+0000 7f0d8f650700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928400 con 0x55a783755000 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(70 notify.6 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920795 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387fd40 con 0x55a783755000 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d96eb1a80 20 handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): orig_sources= new_sources= 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d96eb1a80 20 handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): potential sources added=[] removed=[] 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d96eb1a80 20 handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): orig_dests= new_dests= 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d96eb1a80 20 handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): potential dests added=[] removed=[] 2024-04-27T04:35:43.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.575+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6400 con 0x55a7833c5800 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(71 bar.buckets [call] v33'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 20 rados->read ofs=0 len=0 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6800 con 0x55a7833c5800 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(72 myfoo6 [call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv16 ondisk = 0) v8 ==== 234+0+289 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 20 rados_obj.operate() r=0 bl.length=166 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x13 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.576+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6c00 con 0x55a7833c5800 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(73 myfoo6 [delete,create,call,call,writefull 0~166] v33'17 uv17 ondisk = 0) v8 ==== 318+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d96eb1a80 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T04:35:43.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 94177954277632 in=352b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7800 con 0x55a783755400 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== watch-notify(notify (1) cookie 94177952680064 notify 141733920796 ret 0) v3 ==== 382+0+0 (crc 0 0 0) 0x55a78387da00 con 0x55a783755400 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(74 notify.3 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d8f650700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920796 cookie 94177952680064 notifier 4796 bl.length()=340 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d8f650700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d8f650700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d8f650700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928000 con 0x55a783755400 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920796 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387b040 con 0x55a783755400 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.578+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:76 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381dc00 con 0x55a7833c5800 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.579+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(76 bud.buckets [call] v33'3 uv3 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.579+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:77 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d800 con 0x55a7833c5800 2024-04-27T04:35:43.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.580+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(77 myfoo6 [delete,create,call,writefull 0~166] v33'18 uv18 ondisk = 0) v8 ==== 276+0+0 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.580+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.580+0000 7f0d96eb1a80 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.580+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.580+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [notify cookie 94177954277632 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a783755400 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== watch-notify(notify (1) cookie 94177952680064 notify 141733920797 ret 0) v3 ==== 358+0+0 (crc 0 0 0) 0x55a78387b040 con 0x55a783755400 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d8ee4f700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920797 cookie 94177952680064 notifier 4796 bl.length()=316 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(78 notify.3 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d8ee4f700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d8ee4f700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d8ee4f700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:79 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7000 con 0x55a783755400 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920797 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387b1e0 con 0x55a783755400 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(79 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 4.f 4:f395ffb4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7833c5c00 con 0x55a7833c5800 2024-04-27T04:35:43.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(80 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.581+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 4.36 4:6f3c98dd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928000 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.582+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(81 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.582+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 4.38 4:1f62a13f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928c00 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.582+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(82 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.582+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:83 4.28 4:153413f6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929000 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.582+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(83 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810b40 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.582+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:84 4.3f 4:fd62470d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929400 con 0x55a783755400 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.583+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(84 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.583+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 4.26 4:677ab003:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929800 con 0x55a783755000 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.583+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(85 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810b40 con 0x55a783755000 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.583+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 4.21 4:869f5d7c:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929c00 con 0x55a783755000 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.584+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(86 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.584+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:87 4.38 4:1c9a7278:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a7833c5800 2024-04-27T04:35:43.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.584+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(87 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810d80 con 0x55a7833c5800 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.584+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 4.12 4:4afd3883:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7400 con 0x55a7833c5800 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.584+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(88 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55a783810d80 con 0x55a7833c5800 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.584+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:89 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7800 con 0x55a783755000 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(89 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [call out=377b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+377 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 10 cls_bucket_list_unordered: got thirteen 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 4.1d 4:ba7e5be2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928000 con 0x55a783755c00 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(90 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 get_obj_state: rctx=0x55a7837feb60 obj=myfoo6:thirteen state=0x55a7839381e8 s->prefetch_data=0 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:91 3.2fs0 3:f5593e88:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7833c5c00 con 0x55a783755c00 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(91 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen [getxattrs out=854b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+870 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 10 manifest: total_size = 8 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.3866818395212281031 2024-04-27T04:35:43.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.acl 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.content_type 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.etag 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.idtag 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.manifest 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.source_zone 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 get_obj_state: rctx=0x55a7837feb60 obj=myfoo6:thirteen state=0x55a7839381e8 s->prefetch_data=0 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 : hit (requested=0x17, cached=0x17) 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.585+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:92 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381dc00 con 0x55a783755000 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.587+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(92 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [stat,call,call] v33'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.587+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:93 3.2fs0 3:f5593e88:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen:head [cmpxattr user.rgw.idtag (62) op 1 mode 1 in=76b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381d800 con 0x55a783755c00 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.588+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(93 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen [cmpxattr (62) op 1 mode 1,setxattr (117),setxattr (33)] v33'2 uv2 ondisk = 0) v8 ==== 280+0+0 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.588+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:94 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6800 con 0x55a783755000 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:1 objects processed in :myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]). Next marker thirteen 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.588+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:95 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6c00 con 0x55a783755400 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(95 bud [delete,create,call,writefull 0~346] v33'2 uv2 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d96eb1a80 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify cookie 94177954277632 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7c00 con 0x55a783755000 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== osd_op_reply(94 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [stat,call,call] v33'8 uv8 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d8ee4f700 20 handle_completion(): completion ok for obj=thirteen 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== watch-notify(notify (1) cookie 94177952682368 notify 141733920796 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55a78387fd40 con 0x55a783755000 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d8f650700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920796 cookie 94177952682368 notifier 4796 bl.length()=522 2024-04-27T04:35:43.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== osd_op_reply(96 notify.5 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d8f650700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d8f650700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d8f650700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928400 con 0x55a783755000 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(97 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 33 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920796 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387e340 con 0x55a783755000 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.590+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:98 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7833c5c00 con 0x55a7833c5800 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.591+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(98 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v33'2 uv2 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a7833c5800 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.591+0000 7f0d96eb1a80 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.591+0000 7f0d96eb1a80 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.591+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.591+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify cookie 94177954277632 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928c00 con 0x55a783755000 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 34 ==== watch-notify(notify (1) cookie 94177952682368 notify 141733920797 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55a78387e340 con 0x55a783755000 2024-04-27T04:35:43.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d8f650700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920797 cookie 94177952682368 notifier 4796 bl.length()=177 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d8f650700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d8f650700 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 35 ==== osd_op_reply(99 notify.5 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d8f650700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928800 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 36 ==== osd_op_reply(100 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 37 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920797 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387e4e0 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d96eb1a80 10 removing user index: bar 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.592+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929000 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 38 ==== osd_op_reply(101 bar [delete] v33'4 uv4 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d96eb1a80 10 removing default.rgw.meta+users.uid+bar from cache 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d96eb1a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify cookie 94177954277632 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929800 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 39 ==== watch-notify(notify (1) cookie 94177952684672 notify 141733920798 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55a78387e4e0 con 0x55a783755000 2024-04-27T04:35:43.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d8f650700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920798 cookie 94177952684672 notifier 4796 bl.length()=152 2024-04-27T04:35:43.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 40 ==== osd_op_reply(102 notify.7 [notify cookie 94177954277632 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d8f650700 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783948000 con 0x55a783755000 2024-04-27T04:35:43.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 41 ==== osd_op_reply(103 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.618 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.593+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 42 ==== watch-notify(notify_complete (2) cookie 94177954277632 notify 141733920798 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a78387e680 con 0x55a783755000 2024-04-27T04:35:43.618 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "bud", 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Bar", 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "", 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "bud", 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.594+0000 7f0d7be29700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.594+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:104 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94177954275328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783929c00 con 0x55a7833c5800 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.595+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 34 ==== osd_op_reply(104 data_loggenerations_metadata [watch unwatch cookie 94177954275328] v33'219 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a7833c5800 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.595+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94177952676608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a783928000 con 0x55a783755400 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.596+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94177952676608] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.596+0000 7f0d96eb1a80 20 remove_watcher() i=0 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.596+0000 7f0d96eb1a80 2 removed watcher, disabling cache 2024-04-27T04:35:43.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.596+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94177952677760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c800 con 0x55a7833c5800 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.597+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 35 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94177952677760] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a7833c5800 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.597+0000 7f0d96eb1a80 20 remove_watcher() i=1 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.597+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94177952678912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381dc00 con 0x55a783755c00 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.598+0000 7f0d94e5b700 1 -- 172.21.15.191:0/2122617681 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94177952678912] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811200 con 0x55a783755c00 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.599+0000 7f0d96eb1a80 20 remove_watcher() i=2 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.599+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94177952680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381c400 con 0x55a783755400 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.599+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94177952680064] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783811440 con 0x55a783755400 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.599+0000 7f0d96eb1a80 20 remove_watcher() i=3 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.600+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94177952681216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a78381cc00 con 0x55a7833c5800 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.600+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.1 v2:172.21.15.191:6808/41886 36 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94177952681216] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a7833c5800 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.600+0000 7f0d96eb1a80 20 remove_watcher() i=4 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.600+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94177952682368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7400 con 0x55a783755000 2024-04-27T04:35:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.601+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 43 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94177952682368] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.601+0000 7f0d96eb1a80 20 remove_watcher() i=5 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.601+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94177952683520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b7800 con 0x55a783755000 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.603+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 44 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94177952683520] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.603+0000 7f0d96eb1a80 20 remove_watcher() i=6 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.603+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94177952684672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a7838b6800 con 0x55a783755000 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d9565c700 1 -- 172.21.15.191:0/2122617681 <== osd.0 v2:172.21.15.138:6800/39496 45 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94177952684672] v33'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a783810d80 con 0x55a783755000 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 20 remove_watcher() i=7 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a783755000 msgr2=0x55a7836af180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55a783755000 0x55a7836af180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a7833c5800 msgr2=0x55a7836adb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55a7833c5800 0x55a7836adb80 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a783755400 msgr2=0x55a7836aec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55a783755400 0x55a7836aec00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a783755c00 msgr2=0x55a7836ae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55a783755c00 0x55a7836ae100 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754c00 msgr2=0x55a7836ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754c00 0x55a7836ad600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55a7836a7890 tx=0x55a7834b5800 comp rx=0 tx=0).stop 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 msgr2=0x55a7836acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/2122617681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c5400 0x55a7836acb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55a7842424b0 tx=0x55a783710d20 comp rx=0 tx=0).stop 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 reap_dead start 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d9465a700 1 -- 172.21.15.191:0/2122617681 reap_dead start 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 shutdown_connections 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 >> 172.21.15.191:0/2122617681 conn(0x55a7833c4c00 msgr2=0x55a7834b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 shutdown_connections 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/2122617681 wait complete. 2024-04-27T04:35:43.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a7838b6000 msgr2=0x55a7836af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a7838b6000 0x55a7836af700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a7837abd70 tx=0x55a7838045a0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 msgr2=0x55a7836ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55a783832210 tx=0x55a7838048a0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 shutdown_connections 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a7838b6000 0x55a7836af700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/1133112713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78381d400 0x55a7836ae680 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 >> 172.21.15.191:0/1133112713 conn(0x55a78381d000 msgr2=0x55a7834b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 shutdown_connections 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.604+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/1133112713 wait complete. 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754000 msgr2=0x55a7836ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754000 0x55a7836ad080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55a784242ab0 tx=0x55a783710240 comp rx=0 tx=0).stop 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 msgr2=0x55a7836ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:43.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55a78419da70 tx=0x55a7834b5a40 comp rx=0 tx=0).stop 2024-04-27T04:35:43.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 shutdown_connections 2024-04-27T04:35:43.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55a783754000 0x55a7836ad080 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 --2- 172.21.15.191:0/3328925473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7833c4800 0x55a7836ac580 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:43.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 >> 172.21.15.191:0/3328925473 conn(0x55a7833c4400 msgr2=0x55a7834b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:43.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 shutdown_connections 2024-04-27T04:35:43.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:43.605+0000 7f0d96eb1a80 1 -- 172.21.15.191:0/3328925473 wait complete. 2024-04-27T04:35:43.635 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:35:48.643 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:48.643 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-27T04:35:48.643 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-27T04:35:48.643 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-27T04:35:48.643 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-04-27T04:35:48.704 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:48.705 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b839bca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f9bf792c00 0x55f9bfa7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b839bca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f9bfabd500 con 0x55f9bf792800 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b839bca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f9bf834a00 con 0x55f9bf792c00 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b81165700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f9bf792c00 0x55f9bfa7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b81966700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55f9bf792800 0x55f9bf87e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:47986/0 (socket says 172.21.15.191:47986) 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b81966700 1 -- 172.21.15.191:0/488098115 learned_addr learned my addr 172.21.15.191:0/488098115 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:48.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b80964700 1 -- 172.21.15.191:0/488098115 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870601796 0 0) 0x55f9c0560000 con 0x55f9bf792800 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.704+0000 7f7b80964700 1 -- 172.21.15.191:0/488098115 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f9bfabd680 con 0x55f9bf792800 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b81165700 1 -- 172.21.15.191:0/488098115 >> v1:172.21.15.138:6789/0 conn(0x55f9bf792800 legacy=0x55f9bf87e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b81165700 1 -- 172.21.15.191:0/488098115 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7bbd40 con 0x55f9bf792c00 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b81165700 1 --2- 172.21.15.191:0/488098115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792c00 0x55f9bfa7a580 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x55f9bfa75890 tx=0x55f9bf883980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c82518ded7e6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b80964700 1 -- 172.21.15.191:0/488098115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b4a0 con 0x55f9bf792c00 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b80964700 1 -- 172.21.15.191:0/488098115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bfabd680 con 0x55f9bf792c00 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b80964700 1 -- 172.21.15.191:0/488098115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b600 con 0x55f9bf792c00 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b839bca80 1 -- 172.21.15.191:0/488098115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792c00 msgr2=0x55f9bfa7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.705+0000 7f7b839bca80 1 --2- 172.21.15.191:0/488098115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792c00 0x55f9bfa7a580 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x55f9bfa75890 tx=0x55f9bf883980 comp rx=0 tx=0).stop 2024-04-27T04:35:48.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 -- 172.21.15.191:0/488098115 shutdown_connections 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 --2- 172.21.15.191:0/488098115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792c00 0x55f9bfa7a580 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 -- 172.21.15.191:0/488098115 >> 172.21.15.191:0/488098115 conn(0x55f9bf792400 msgr2=0x55f9bf87e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 -- 172.21.15.191:0/488098115 shutdown_connections 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 -- 172.21.15.191:0/488098115 wait complete. 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.710 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.706+0000 7f7b839bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bf834500 con 0x55f9bf792800 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b81966700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b81966700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49302/0 (socket says 172.21.15.191:49302) 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b81966700 1 -- 172.21.15.191:0/1116116281 learned_addr learned my addr 172.21.15.191:0/1116116281 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b81966700 1 -- 172.21.15.191:0/1116116281 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7bbba0 con 0x55f9bf792800 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b81966700 1 --2- 172.21.15.191:0/1116116281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x55f9c0563d70 tx=0x55f9bf883b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35562eb184cf0dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b80163700 1 -- 172.21.15.191:0/1116116281 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b1e0 con 0x55f9bf792800 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b80163700 1 -- 172.21.15.191:0/1116116281 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bfabdb00 con 0x55f9bf792800 2024-04-27T04:35:48.747 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.707+0000 7f7b80163700 1 -- 172.21.15.191:0/1116116281 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b4a0 con 0x55f9bf792800 2024-04-27T04:35:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- 172.21.15.191:0/1116116281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 msgr2=0x55f9bfa7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1116116281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x55f9c0563d70 tx=0x55f9bf883b60 comp rx=0 tx=0).stop 2024-04-27T04:35:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- 172.21.15.191:0/1116116281 shutdown_connections 2024-04-27T04:35:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1116116281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.750 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- 172.21.15.191:0/1116116281 >> 172.21.15.191:0/1116116281 conn(0x55f9bf792400 msgr2=0x55f9bf87f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- 172.21.15.191:0/1116116281 shutdown_connections 2024-04-27T04:35:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- 172.21.15.191:0/1116116281 wait complete. 2024-04-27T04:35:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.708+0000 7f7b839bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bf8343c0 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b81966700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b81966700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49306/0 (socket says 172.21.15.191:49306) 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b81966700 1 -- 172.21.15.191:0/1146010520 learned_addr learned my addr 172.21.15.191:0/1146010520 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b81966700 1 -- 172.21.15.191:0/1146010520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7bba00 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b81966700 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x55f9c0563e00 tx=0x55f9bf883920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b7e960700 1 -- 172.21.15.191:0/1146010520 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b600 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b7e960700 1 -- 172.21.15.191:0/1146010520 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bf833c80 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b7e960700 1 -- 172.21.15.191:0/1146010520 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b1e0 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.709+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f9bf7bb860 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.710+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9bf7bb6c0 con 0x55f9bf792800 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.710+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.710+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.710+0000 7f7b839bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.710+0000 7f7b839bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bf7ce640 con 0x55f9bf793400 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.710+0000 7f7b7e960700 1 -- 172.21.15.191:0/1146010520 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f9c0534b00 con 0x55f9bf792800 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b7e960700 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1e000 0x55f9bfa7b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b82167700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b82167700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49318/0 (socket says 172.21.15.191:49318) 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b82167700 1 -- 172.21.15.191:0/3586009163 learned_addr learned my addr 172.21.15.191:0/3586009163 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b7e960700 1 -- 172.21.15.191:0/1146010520 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f9bf80d180 con 0x55f9bf792800 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b81966700 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1e000 0x55f9bfa7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b82167700 1 -- 172.21.15.191:0/3586009163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7ba000 con 0x55f9bf793400 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b81966700 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1e000 0x55f9bfa7b080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55f9c0628f30 tx=0x55f9bfade240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b82167700 1 --2- 172.21.15.191:0/3586009163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x55f9c0563f50 tx=0x55f9bfadede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a708dc1c5134ac9e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b7c15b700 1 -- 172.21.15.191:0/3586009163 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b600 con 0x55f9bf793400 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b7c15b700 1 -- 172.21.15.191:0/3586009163 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bfabc480 con 0x55f9bf793400 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.711+0000 7f7b7c15b700 1 -- 172.21.15.191:0/3586009163 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b1e0 con 0x55f9bf793400 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- 172.21.15.191:0/3586009163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 msgr2=0x55f9bfa7ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 --2- 172.21.15.191:0/3586009163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x55f9c0563f50 tx=0x55f9bfadede0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- 172.21.15.191:0/3586009163 shutdown_connections 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 --2- 172.21.15.191:0/3586009163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- 172.21.15.191:0/3586009163 >> 172.21.15.191:0/3586009163 conn(0x55f9bf792c00 msgr2=0x55f9bf87f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- 172.21.15.191:0/3586009163 shutdown_connections 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- 172.21.15.191:0/3586009163 wait complete. 2024-04-27T04:35:48.754 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.712+0000 7f7b839bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bf7ce8c0 con 0x55f9bf793400 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b82167700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b82167700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49324/0 (socket says 172.21.15.191:49324) 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 learned_addr learned my addr 172.21.15.191:0/1436401555 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7baea0 con 0x55f9bf793400 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b82167700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x55f9c0628420 tx=0x55f9bfade840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b7a958700 1 -- 172.21.15.191:0/1436401555 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84ba20 con 0x55f9bf793400 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b7a958700 1 -- 172.21.15.191:0/1436401555 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bfabc180 con 0x55f9bf793400 2024-04-27T04:35:48.755 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f9bf7ba820 con 0x55f9bf793400 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b7a958700 1 -- 172.21.15.191:0/1436401555 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b600 con 0x55f9bf793400 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.713+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9bf7ba9c0 con 0x55f9bf793400 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.714+0000 7f7b7a958700 1 -- 172.21.15.191:0/1436401555 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f9c0534dc0 con 0x55f9bf793400 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.714+0000 7f7b7a958700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1ec00 0x55f9bfa7b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.714+0000 7f7b7a958700 1 -- 172.21.15.191:0/1436401555 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f9bf80d340 con 0x55f9bf793400 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.714+0000 7f7b81165700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1ec00 0x55f9bfa7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.714+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f9bf793800 0x55f9bfa7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bf793c00 con 0x55f9bf793800 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b82167700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f9bf793800 0x55f9bfa7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b81165700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1ec00 0x55f9bfa7b600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f9bfa75890 tx=0x55f9bf883800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.756 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b82167700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f9bf793800 0x55f9bfa7bb80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b839bca80 20 realm 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.715+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f000 con 0x55f9bf793800 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f400 con 0x55f9bf793800 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.757 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f800 con 0x55f9bf793800 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f9bfb1fc00 0x55f9bfa7c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.716+0000 7f7b81966700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f9bfb1fc00 0x55f9bfa7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b81966700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f9bfb1fc00 0x55f9bfa7c100 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b839bca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:48.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b839bca80 20 searching for the correct realm 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f400 con 0x55f9bfb1fc00 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b81966700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f9bfb1f400 0x55f9bfa7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.717+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f000 con 0x55f9bfb1f400 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.718+0000 7f7b81165700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f9bfb1f400 0x55f9bfa7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.718+0000 7f7b81165700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f9bfb1f400 0x55f9bfa7cc00 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.718+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.718+0000 7f7b81165700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f9bfb1f000 0x55f9bfa7d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.718+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.718+0000 7f7b82167700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f9bfb1f000 0x55f9bfa7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f9bfb1f000 0x55f9bfa7d180 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.719+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.720+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.720+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.720+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.720+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f400 2024-04-27T04:35:48.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.720+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.720+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f400 2024-04-27T04:35:48.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.721+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.722+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.723+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.723+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.723+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.723+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.723+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.723+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f400 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.724+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f400 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f400 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.725+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f400 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.726+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bf793c00 con 0x55f9bf793800 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6400 con 0x55f9bf793800 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6800 con 0x55f9bfb1fc00 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 zone default found 2024-04-27T04:35:48.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 4 Realm: () 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 zonegroup default 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.727+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6c00 con 0x55f9bfb1f000 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7400 con 0x55f9bfb1fc00 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 started sync module instance, tier type = 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:48.768 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.728+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7800 con 0x55f9bfb1f400 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.729+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.729+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94531150617344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6800 con 0x55f9bfb1f400 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.729+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7000 con 0x55f9bf793800 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.730+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94531150617344] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.731+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.731+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94531150618496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7400 con 0x55f9bf793800 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.731+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6400 con 0x55f9bfb1fc00 2024-04-27T04:35:48.769 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.732+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.732+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94531150619648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6c00 con 0x55f9bfb1fc00 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.732+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bf793c00 con 0x55f9bfb1f400 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.732+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94531150618496] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.733+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94531150619648] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.733+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.733+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94531150620800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfb1f800 con 0x55f9bfb1f400 2024-04-27T04:35:48.770 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.733+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bf793800 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.734+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94531150620800] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.734+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.734+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94531150621952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7800 con 0x55f9bf793800 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.734+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6800 con 0x55f9bfb1f000 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.735+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94531150621952] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.736+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.736+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94531150623104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7c00 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.736+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7000 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.737+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94531150623104] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.737+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.737+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94531150624256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7400 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.737+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6000 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94531150624256] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94531150625408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7800 con 0x55f9bfb1f000 2024-04-27T04:35:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=0 2024-04-27T04:35:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=1 2024-04-27T04:35:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=2 2024-04-27T04:35:48.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=3 2024-04-27T04:35:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=4 2024-04-27T04:35:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=5 2024-04-27T04:35:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.738+0000 7f7b839bca80 20 add_watcher() i=6 2024-04-27T04:35:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.739+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94531150625408] v33'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.739+0000 7f7b839bca80 20 add_watcher() i=7 2024-04-27T04:35:48.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.739+0000 7f7b839bca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:48.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.739+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6800 con 0x55f9bf793800 2024-04-27T04:35:48.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.740+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.740+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94531152191488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7c00 con 0x55f9bf793800 2024-04-27T04:35:48.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.741+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94531152191488] v33'220 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.741+0000 7f7b68934700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.741+0000 7f7b839bca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.741+0000 7f7b839bca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.741+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.741+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b839bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b839bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bf8343c0 con 0x55f9bfbe7400 2024-04-27T04:35:48.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b81165700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b81165700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49326/0 (socket says 172.21.15.191:49326) 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b81165700 1 -- 172.21.15.191:0/2600964598 learned_addr learned my addr 172.21.15.191:0/2600964598 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b81165700 1 -- 172.21.15.191:0/2600964598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7bab60 con 0x55f9bfbe7400 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b81165700 1 --2- 172.21.15.191:0/2600964598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x55f9bfb87830 tx=0x55f9bf8839e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4092076166d8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b68133700 1 -- 172.21.15.191:0/2600964598 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bf84b760 con 0x55f9bfbe7400 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.742+0000 7f7b68133700 1 -- 172.21.15.191:0/2600964598 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bfc0a000 con 0x55f9bfbe7400 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b68133700 1 -- 172.21.15.191:0/2600964598 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bfc0c000 con 0x55f9bfbe7400 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 -- 172.21.15.191:0/2600964598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 msgr2=0x55f9bfa7c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 --2- 172.21.15.191:0/2600964598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x55f9bfb87830 tx=0x55f9bf8839e0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 -- 172.21.15.191:0/2600964598 shutdown_connections 2024-04-27T04:35:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 --2- 172.21.15.191:0/2600964598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 -- 172.21.15.191:0/2600964598 >> 172.21.15.191:0/2600964598 conn(0x55f9bfbe7000 msgr2=0x55f9bf880000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 -- 172.21.15.191:0/2600964598 shutdown_connections 2024-04-27T04:35:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.743+0000 7f7b839bca80 1 -- 172.21.15.191:0/2600964598 wait complete. 2024-04-27T04:35:48.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b839bca80 1 Processor -- start 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b839bca80 1 -- start start 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b839bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b839bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bf834500 con 0x55f9bfbe7400 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b81165700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b81165700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49336/0 (socket says 172.21.15.191:49336) 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b81165700 1 -- 172.21.15.191:0/1145922743 learned_addr learned my addr 172.21.15.191:0/1145922743 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b81165700 1 -- 172.21.15.191:0/1145922743 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bf7bad00 con 0x55f9bfbe7400 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.744+0000 7f7b81165700 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x55f9bfbfb260 tx=0x55f9bfbc88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.745+0000 7f7b66930700 1 -- 172.21.15.191:0/1145922743 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bfc0c160 con 0x55f9bfbe7400 2024-04-27T04:35:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.745+0000 7f7b66930700 1 -- 172.21.15.191:0/1145922743 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9bfc0a180 con 0x55f9bfbe7400 2024-04-27T04:35:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.745+0000 7f7b66930700 1 -- 172.21.15.191:0/1145922743 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f9bfc0c2c0 con 0x55f9bfbe7400 2024-04-27T04:35:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.745+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f9bf7ba340 con 0x55f9bfbe7400 2024-04-27T04:35:48.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.745+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9bf7bbd40 con 0x55f9bfbe7400 2024-04-27T04:35:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.746+0000 7f7b66930700 1 -- 172.21.15.191:0/1145922743 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f9c0535b80 con 0x55f9bfbe7400 2024-04-27T04:35:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.746+0000 7f7b66930700 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfc84000 0x55f9bfa7d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.746+0000 7f7b66930700 1 -- 172.21.15.191:0/1145922743 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f9bf80d500 con 0x55f9bfbe7400 2024-04-27T04:35:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.746+0000 7f7b82167700 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfc84000 0x55f9bfa7d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.746+0000 7f7b839bca80 5 note: GC not initialized 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.746+0000 7f7b82167700 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfc84000 0x55f9bfa7d700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55f9bfb87320 tx=0x55f9bfbc85a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b6412b700 20 reqs_thread_entry: start 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b839bca80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b839bca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b62127700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:19 2024 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b61905700 20 reqs_thread_entry: start 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b839bca80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.747+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe7800 con 0x55f9bfb1f000 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6800 con 0x55f9bfb1f000 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:35:48.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.748+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc84400 con 0x55f9bfb1f000 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v33'5 uv5 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b839bca80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b839bca80 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b839bca80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94531152193792 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc84800 con 0x55f9bfb1f000 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== watch-notify(notify (1) cookie 94531150623104 notify 141733920799 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55f9bfb89d40 con 0x55f9bfb1f000 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(65 notify.5 [notify cookie 94531152193792 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.749+0000 7f7b7c15b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920799 cookie 94531150623104 notifier 4804 bl.length()=522 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.750+0000 7f7b7c15b700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.750+0000 7f7b7c15b700 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.750+0000 7f7b7c15b700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfcf4000 con 0x55f9bfb1f000 2024-04-27T04:35:48.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.750+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.750+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== watch-notify(notify_complete (2) cookie 94531152193792 notify 141733920799 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f9bfc4dba0 con 0x55f9bfb1f000 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.750+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc84c00 con 0x55f9bfb1f000 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v33'9 uv9 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b839bca80 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b839bca80 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b839bca80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94531152193792 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc85000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94531150619648 notify 141733920776 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55f9bfadbd40 con 0x55f9bfb1fc00 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b7c15b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920776 cookie 94531150619648 notifier 4804 bl.length()=177 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b7c15b700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b7c15b700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(68 notify.2 [notify cookie 94531152193792 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.751+0000 7f7b7c15b700 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfcf4400 con 0x55f9bfb1fc00 2024-04-27T04:35:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.752+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.752+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== watch-notify(notify_complete (2) cookie 94531152193792 notify 141733920776 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f9bfc4ba00 con 0x55f9bfb1fc00 2024-04-27T04:35:48.790 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:35:48.790 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "fud", 2024-04-27T04:35:48.790 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Fud", 2024-04-27T04:35:48.790 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "", 2024-04-27T04:35:48.791 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:35:48.791 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 4, 2024-04-27T04:35:48.791 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:35:48.791 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:35:48.791 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:35:48.791 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "fud", 2024-04-27T04:35:48.792 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T04:35:48.792 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T04:35:48.792 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:35:48.792 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:35:48.792 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [], 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:48.793 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:35:48.794 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:35:48.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.752+0000 7f7b68934700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.752+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94531152191488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc85400 con 0x55f9bf793800 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.753+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94531152191488] v33'221 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.753+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94531150617344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc85800 con 0x55f9bfb1f400 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.754+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94531150617344] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.755+0000 7f7b839bca80 20 remove_watcher() i=0 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.755+0000 7f7b839bca80 2 removed watcher, disabling cache 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.755+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94531150618496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc85c00 con 0x55f9bf793800 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.756+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94531150618496] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.756+0000 7f7b839bca80 20 remove_watcher() i=1 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.756+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94531150619648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc85000 con 0x55f9bfb1fc00 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.757+0000 7f7b81966700 1 -- 172.21.15.191:0/1436401555 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94531150619648] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf200 con 0x55f9bfb1fc00 2024-04-27T04:35:48.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.757+0000 7f7b839bca80 20 remove_watcher() i=2 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.757+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94531150620800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc84400 con 0x55f9bfb1f400 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.758+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94531150620800] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdf440 con 0x55f9bfb1f400 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.758+0000 7f7b839bca80 20 remove_watcher() i=3 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.758+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94531150621952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc84800 con 0x55f9bf793800 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.759+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94531150621952] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bf793800 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.760+0000 7f7b839bca80 20 remove_watcher() i=4 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.760+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94531150623104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfc84c00 con 0x55f9bfb1f000 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.761+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94531150623104] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.761+0000 7f7b839bca80 20 remove_watcher() i=5 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.761+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94531150624256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6400 con 0x55f9bfb1f000 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.762+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94531150624256] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.762+0000 7f7b839bca80 20 remove_watcher() i=6 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.762+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94531150625408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f9bfbe6c00 con 0x55f9bfb1f000 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b82167700 1 -- 172.21.15.191:0/1436401555 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94531150625408] v33'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f9bfbdeb40 con 0x55f9bfb1f000 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 20 remove_watcher() i=7 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f9bfb1f000 msgr2=0x55f9bfa7d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f9bfb1f000 0x55f9bfa7d180 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f9bf793800 msgr2=0x55f9bfa7bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f9bf793800 0x55f9bfa7bb80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f9bfb1f400 msgr2=0x55f9bfa7cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f9bfb1f400 0x55f9bfa7cc00 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f9bfb1fc00 msgr2=0x55f9bfa7c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f9bfb1fc00 0x55f9bfa7c100 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1ec00 msgr2=0x55f9bfa7b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1ec00 0x55f9bfa7b600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f9bfa75890 tx=0x55f9bf883800 comp rx=0 tx=0).stop 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 msgr2=0x55f9bfa7ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1436401555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf793400 0x55f9bfa7ab00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x55f9c0628420 tx=0x55f9bfade840 comp rx=0 tx=0).stop 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 reap_dead start 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b81165700 1 -- 172.21.15.191:0/1436401555 reap_dead start 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 shutdown_connections 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 >> 172.21.15.191:0/1436401555 conn(0x55f9bf792c00 msgr2=0x55f9bf87f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 shutdown_connections 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.763+0000 7f7b839bca80 1 -- 172.21.15.191:0/1436401555 wait complete. 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfc84000 msgr2=0x55f9bfa7d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfc84000 0x55f9bfa7d700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55f9bfb87320 tx=0x55f9bfbc85a0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 msgr2=0x55f9bfa7c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x55f9bfbfb260 tx=0x55f9bfbc88a0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 shutdown_connections 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfc84000 0x55f9bfa7d700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1145922743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bfbe7400 0x55f9bfa7c680 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 >> 172.21.15.191:0/1145922743 conn(0x55f9bfbe7000 msgr2=0x55f9bf880000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 shutdown_connections 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1145922743 wait complete. 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1e000 msgr2=0x55f9bfa7b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1e000 0x55f9bfa7b080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55f9c0628f30 tx=0x55f9bfade240 comp rx=0 tx=0).stop 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 msgr2=0x55f9bfa7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x55f9c0563e00 tx=0x55f9bf883920 comp rx=0 tx=0).stop 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 shutdown_connections 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f9bfb1e000 0x55f9bfa7b080 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 --2- 172.21.15.191:0/1146010520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bf792800 0x55f9bfa7a580 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 >> 172.21.15.191:0/1146010520 conn(0x55f9bf792400 msgr2=0x55f9bf87f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 shutdown_connections 2024-04-27T04:35:48.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.764+0000 7f7b839bca80 1 -- 172.21.15.191:0/1146010520 wait complete. 2024-04-27T04:35:48.808 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:35:48.820 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:48.824 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-04-27T04:35:48.825 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-27T04:35:48.825 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-27T04:35:48.825 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-04-27T04:35:48.872 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:48.873 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:48.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b366fa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ae08db6c00 0x55ae0909e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b366fa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ae090e1500 con 0x55ae08db6800 2024-04-27T04:35:48.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b366fa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ae08e58a00 con 0x55ae08db6c00 2024-04-27T04:35:48.876 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b0e18700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ae08db6c00 0x55ae0909e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b1619700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55ae08db6800 0x55ae08ea2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:47988/0 (socket says 172.21.15.191:47988) 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b0e18700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ae08db6c00 0x55ae0909e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49352/0 (socket says 172.21.15.191:49352) 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b1619700 1 -- 172.21.15.191:0/870338890 learned_addr learned my addr 172.21.15.191:0/870338890 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b0e18700 1 -- 172.21.15.191:0/870338890 >> v1:172.21.15.138:6789/0 conn(0x55ae08db6800 legacy=0x55ae08ea2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.872+0000 7f92b0e18700 1 -- 172.21.15.191:0/870338890 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08ddfd40 con 0x55ae08db6c00 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b0e18700 1 --2- 172.21.15.191:0/870338890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6c00 0x55ae0909e580 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55ae09099890 tx=0x55ae08ea7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d852b171d8b80f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b0617700 1 -- 172.21.15.191:0/870338890 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f4a0 con 0x55ae08db6c00 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b0617700 1 -- 172.21.15.191:0/870338890 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae090e1800 con 0x55ae08db6c00 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b0617700 1 -- 172.21.15.191:0/870338890 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f600 con 0x55ae08db6c00 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 -- 172.21.15.191:0/870338890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6c00 msgr2=0x55ae0909e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 --2- 172.21.15.191:0/870338890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6c00 0x55ae0909e580 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55ae09099890 tx=0x55ae08ea7920 comp rx=0 tx=0).stop 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 -- 172.21.15.191:0/870338890 shutdown_connections 2024-04-27T04:35:48.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 --2- 172.21.15.191:0/870338890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6c00 0x55ae0909e580 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 -- 172.21.15.191:0/870338890 >> 172.21.15.191:0/870338890 conn(0x55ae08db6400 msgr2=0x55ae08ea2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 -- 172.21.15.191:0/870338890 shutdown_connections 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.873+0000 7f92b366fa80 1 -- 172.21.15.191:0/870338890 wait complete. 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b366fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b366fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae08e58500 con 0x55ae08db6800 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b1619700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b1619700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49366/0 (socket says 172.21.15.191:49366) 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.874+0000 7f92b1619700 1 -- 172.21.15.191:0/3954085107 learned_addr learned my addr 172.21.15.191:0/3954085107 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b1619700 1 -- 172.21.15.191:0/3954085107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08ddfba0 con 0x55ae08db6800 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b1619700 1 --2- 172.21.15.191:0/3954085107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55ae09b8f890 tx=0x55ae08ea7b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c580f67c1614cc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92afe16700 1 -- 172.21.15.191:0/3954085107 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f4a0 con 0x55ae08db6800 2024-04-27T04:35:48.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92afe16700 1 -- 172.21.15.191:0/3954085107 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae090e1b00 con 0x55ae08db6800 2024-04-27T04:35:48.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92afe16700 1 -- 172.21.15.191:0/3954085107 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f600 con 0x55ae08db6800 2024-04-27T04:35:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 -- 172.21.15.191:0/3954085107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 msgr2=0x55ae0909e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3954085107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55ae09b8f890 tx=0x55ae08ea7b60 comp rx=0 tx=0).stop 2024-04-27T04:35:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 -- 172.21.15.191:0/3954085107 shutdown_connections 2024-04-27T04:35:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3954085107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 -- 172.21.15.191:0/3954085107 >> 172.21.15.191:0/3954085107 conn(0x55ae08db6400 msgr2=0x55ae08ea3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 -- 172.21.15.191:0/3954085107 shutdown_connections 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.875+0000 7f92b366fa80 1 -- 172.21.15.191:0/3954085107 wait complete. 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b366fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b366fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae08e583c0 con 0x55ae08db6800 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b1619700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b1619700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49372/0 (socket says 172.21.15.191:49372) 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b1619700 1 -- 172.21.15.191:0/4283159287 learned_addr learned my addr 172.21.15.191:0/4283159287 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b1619700 1 -- 172.21.15.191:0/4283159287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08ddfa00 con 0x55ae08db6800 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b1619700 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x55ae09b8fad0 tx=0x55ae08ea7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92ae613700 1 -- 172.21.15.191:0/4283159287 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f8c0 con 0x55ae08db6800 2024-04-27T04:35:48.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92ae613700 1 -- 172.21.15.191:0/4283159287 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae08e57c80 con 0x55ae08db6800 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92ae613700 1 -- 172.21.15.191:0/4283159287 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f4a0 con 0x55ae08db6800 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.876+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae08ddf860 con 0x55ae08db6800 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae08ddf6c0 con 0x55ae08db6800 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b366fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b366fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae08df2640 con 0x55ae08db7400 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92ae613700 1 -- 172.21.15.191:0/4283159287 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ae09b58b00 con 0x55ae08db6800 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b1e1a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b1e1a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49376/0 (socket says 172.21.15.191:49376) 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.877+0000 7f92b1e1a700 1 -- 172.21.15.191:0/2241677663 learned_addr learned my addr 172.21.15.191:0/2241677663 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92ae613700 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150000 0x55ae0909f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92ae613700 1 -- 172.21.15.191:0/4283159287 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ae08e31180 con 0x55ae08db6800 2024-04-27T04:35:48.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b1619700 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150000 0x55ae0909f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b1e1a700 1 -- 172.21.15.191:0/2241677663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08dde000 con 0x55ae08db7400 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/2241677663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55ae09c35830 tx=0x55ae09108de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c876baceb3ef583 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b1619700 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150000 0x55ae0909f080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55ae09c34840 tx=0x55ae09108240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92abe0e700 1 -- 172.21.15.191:0/2241677663 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f8c0 con 0x55ae08db7400 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92abe0e700 1 -- 172.21.15.191:0/2241677663 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae090e0300 con 0x55ae08db7400 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92abe0e700 1 -- 172.21.15.191:0/2241677663 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f4a0 con 0x55ae08db7400 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 -- 172.21.15.191:0/2241677663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 msgr2=0x55ae0909eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 --2- 172.21.15.191:0/2241677663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55ae09c35830 tx=0x55ae09108de0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 -- 172.21.15.191:0/2241677663 shutdown_connections 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 --2- 172.21.15.191:0/2241677663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 -- 172.21.15.191:0/2241677663 >> 172.21.15.191:0/2241677663 conn(0x55ae08db6c00 msgr2=0x55ae08ea3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 -- 172.21.15.191:0/2241677663 shutdown_connections 2024-04-27T04:35:48.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.878+0000 7f92b366fa80 1 -- 172.21.15.191:0/2241677663 wait complete. 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b366fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b366fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae08df28c0 con 0x55ae08db7400 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b1e1a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b1e1a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49378/0 (socket says 172.21.15.191:49378) 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 learned_addr learned my addr 172.21.15.191:0/3262643690 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.879+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08ddeea0 con 0x55ae08db7400 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.880+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55ae09c34480 tx=0x55ae09108660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.880+0000 7f92aa60b700 1 -- 172.21.15.191:0/3262643690 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6fb80 con 0x55ae08db7400 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.880+0000 7f92aa60b700 1 -- 172.21.15.191:0/3262643690 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae08e57c80 con 0x55ae08db7400 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.880+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae08dde820 con 0x55ae08db7400 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.880+0000 7f92aa60b700 1 -- 172.21.15.191:0/3262643690 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f8c0 con 0x55ae08db7400 2024-04-27T04:35:48.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.880+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae08dde9c0 con 0x55ae08db7400 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92aa60b700 1 -- 172.21.15.191:0/3262643690 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ae09b58dc0 con 0x55ae08db7400 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92aa60b700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150c00 0x55ae0909f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92aa60b700 1 -- 172.21.15.191:0/3262643690 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ae08e31340 con 0x55ae08db7400 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b0e18700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150c00 0x55ae0909f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ae08db7800 0x55ae0909fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae08db7c00 con 0x55ae08db7800 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ae08db7800 0x55ae0909fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b0e18700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150c00 0x55ae0909f600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55ae09099890 tx=0x55ae08ea7800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.881+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ae08db7800 0x55ae0909fb80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 realm 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151000 con 0x55ae08db7800 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151400 con 0x55ae08db7800 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151800 con 0x55ae08db7800 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.903 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ae09151c00 0x55ae090a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151000 con 0x55ae09151c00 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.882+0000 7f92b1619700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ae09151c00 0x55ae090a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b1619700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ae09151c00 0x55ae090a0100 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b366fa80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b366fa80 20 searching for the correct realm 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151400 con 0x55ae09151c00 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b1619700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ae09151400 0x55ae090a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151000 con 0x55ae09151400 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.883+0000 7f92b0e18700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ae09151400 0x55ae090a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.904 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.884+0000 7f92b0e18700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ae09151400 0x55ae090a0c00 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.884+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.884+0000 7f92b0e18700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ae09151000 0x55ae090a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.884+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.884+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ae09151000 0x55ae090a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.884+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ae09151000 0x55ae090a1180 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.885+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.905 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151400 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.886+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151400 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.887+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.888+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151400 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.889+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151400 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.890+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151400 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151c00 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.891+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151400 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:48.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae08db7c00 con 0x55ae08db7800 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216400 con 0x55ae08db7800 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216800 con 0x55ae09151c00 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:48.910 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 zone default found 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 4 Realm: () 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 zonegroup default 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.892+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216c00 con 0x55ae09151000 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217000 con 0x55ae09151c00 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.911 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217400 con 0x55ae09151c00 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 started sync module instance, tier type = 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.893+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217800 con 0x55ae09151400 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.894+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.894+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94205964278528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216800 con 0x55ae09151400 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.894+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217000 con 0x55ae08db7800 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.895+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94205964278528] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.895+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.895+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94205964279680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217400 con 0x55ae08db7800 2024-04-27T04:35:48.912 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.896+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216400 con 0x55ae09151c00 2024-04-27T04:35:48.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.897+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94205964279680] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.897+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.897+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94205964280832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216c00 con 0x55ae09151c00 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.897+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae08db7c00 con 0x55ae09151400 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.898+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.898+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94205964281984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09151800 con 0x55ae09151400 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.898+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae08db7800 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.898+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94205964280832] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.899+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.899+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94205964281984] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.899+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94205964283136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217800 con 0x55ae08db7800 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.899+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216800 con 0x55ae09151000 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.900+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94205964283136] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.901+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.901+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94205964284288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217c00 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.901+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217000 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.903+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94205964284288] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.903+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.903+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94205964285440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217400 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.903+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216000 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94205964285440] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94205964286592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217800 con 0x55ae09151000 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=0 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=1 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=2 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=3 2024-04-27T04:35:48.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=4 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=5 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.904+0000 7f92b366fa80 20 add_watcher() i=6 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.905+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94205964286592] v33'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.905+0000 7f92b366fa80 20 add_watcher() i=7 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.905+0000 7f92b366fa80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.905+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216800 con 0x55ae08db7800 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.905+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.905+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94205965893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217c00 con 0x55ae08db7800 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94205965893632] v33'222 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92985e7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b366fa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b366fa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.925 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b366fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.906+0000 7f92b366fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae08e583c0 con 0x55ae09217400 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f92b0e18700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f92b0e18700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49384/0 (socket says 172.21.15.191:49384) 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f92b0e18700 1 -- 172.21.15.191:0/1482134593 learned_addr learned my addr 172.21.15.191:0/1482134593 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f92b0e18700 1 -- 172.21.15.191:0/1482134593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08ddeb60 con 0x55ae09217400 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f92b0e18700 1 --2- 172.21.15.191:0/1482134593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55ae091af620 tx=0x55ae08ea7980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d696e71cbfe611fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f9297de6700 1 -- 172.21.15.191:0/1482134593 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae08e6f760 con 0x55ae09217400 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f9297de6700 1 -- 172.21.15.191:0/1482134593 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae09234000 con 0x55ae09217400 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.907+0000 7f9297de6700 1 -- 172.21.15.191:0/1482134593 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae09236000 con 0x55ae09217400 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- 172.21.15.191:0/1482134593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 msgr2=0x55ae090a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 --2- 172.21.15.191:0/1482134593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55ae091af620 tx=0x55ae08ea7980 comp rx=0 tx=0).stop 2024-04-27T04:35:48.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- 172.21.15.191:0/1482134593 shutdown_connections 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 --2- 172.21.15.191:0/1482134593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- 172.21.15.191:0/1482134593 >> 172.21.15.191:0/1482134593 conn(0x55ae09217000 msgr2=0x55ae08ea4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- 172.21.15.191:0/1482134593 shutdown_connections 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- 172.21.15.191:0/1482134593 wait complete. 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 Processor -- start 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- start start 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.908+0000 7f92b366fa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae08e58500 con 0x55ae09217400 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b0e18700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b0e18700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49396/0 (socket says 172.21.15.191:49396) 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b0e18700 1 -- 172.21.15.191:0/553937887 learned_addr learned my addr 172.21.15.191:0/553937887 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b0e18700 1 -- 172.21.15.191:0/553937887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae08dded00 con 0x55ae09217400 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b0e18700 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55ae09228210 tx=0x55ae091fc8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92965e3700 1 -- 172.21.15.191:0/553937887 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae09236160 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92965e3700 1 -- 172.21.15.191:0/553937887 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae09234180 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae08dde340 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92965e3700 1 -- 172.21.15.191:0/553937887 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae092362c0 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.909+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae08ddfd40 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.910+0000 7f92965e3700 1 -- 172.21.15.191:0/553937887 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ae09b59b80 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.910+0000 7f92965e3700 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae092a8000 0x55ae090a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.910+0000 7f92965e3700 1 -- 172.21.15.191:0/553937887 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ae08e31500 con 0x55ae09217400 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.910+0000 7f92b366fa80 5 note: GC not initialized 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.910+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae092a8000 0x55ae090a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f92b1e1a700 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae092a8000 0x55ae090a1700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ae091afd70 tx=0x55ae091fc5a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f9293dde700 20 reqs_thread_entry: start 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f92b366fa80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f92b366fa80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:48.928 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f9291dda700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:19 2024 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f92915b8700 20 reqs_thread_entry: start 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f92b366fa80 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.911+0000 7f92b366fa80 20 rados->read ofs=0 len=0 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09217800 con 0x55ae09151400 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b366fa80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b366fa80 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b366fa80 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b366fa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.912+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae09216800 con 0x55ae09151000 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.913+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 fud [create,call,writefull 0~181] v33'6 uv5 ondisk = -17 ((17) File exists)) v8 ==== 231+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.929 INFO:teuthology.orchestra.run.smithi191.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.914+0000 7f92985e7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.914+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94205965893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a8400 con 0x55ae08db7800 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.915+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94205965893632] v33'223 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.915+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94205964278528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a8800 con 0x55ae09151400 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.916+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94205964278528] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.916+0000 7f92b366fa80 20 remove_watcher() i=0 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.916+0000 7f92b366fa80 2 removed watcher, disabling cache 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.916+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94205964279680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a8c00 con 0x55ae08db7800 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.917+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94205964279680] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.917+0000 7f92b366fa80 20 remove_watcher() i=1 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.917+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94205964280832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a9000 con 0x55ae09151c00 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.918+0000 7f92b1619700 1 -- 172.21.15.191:0/3262643690 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94205964280832] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9200 con 0x55ae09151c00 2024-04-27T04:35:48.930 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.918+0000 7f92b366fa80 20 remove_watcher() i=2 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.918+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94205964281984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a9400 con 0x55ae09151400 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.919+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94205964281984] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f9440 con 0x55ae09151400 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.919+0000 7f92b366fa80 20 remove_watcher() i=3 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.919+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94205964283136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a9800 con 0x55ae08db7800 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.921+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94205964283136] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae08db7800 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.921+0000 7f92b366fa80 20 remove_watcher() i=4 2024-04-27T04:35:48.931 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.921+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94205964284288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a9c00 con 0x55ae09151000 2024-04-27T04:35:48.934 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.922+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94205964284288] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.922+0000 7f92b366fa80 20 remove_watcher() i=5 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.922+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94205964285440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a8800 con 0x55ae09151000 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.923+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94205964285440] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.923+0000 7f92b366fa80 20 remove_watcher() i=6 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.923+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94205964286592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ae092a9400 con 0x55ae09151000 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b1e1a700 1 -- 172.21.15.191:0/3262643690 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94205964286592] v33'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae091f8b40 con 0x55ae09151000 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 20 remove_watcher() i=7 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ae09151000 msgr2=0x55ae090a1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.935 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ae09151000 0x55ae090a1180 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ae08db7800 msgr2=0x55ae0909fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ae08db7800 0x55ae0909fb80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ae09151400 msgr2=0x55ae090a0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ae09151400 0x55ae090a0c00 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ae09151c00 msgr2=0x55ae090a0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ae09151c00 0x55ae090a0100 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150c00 msgr2=0x55ae0909f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150c00 0x55ae0909f600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55ae09099890 tx=0x55ae08ea7800 comp rx=0 tx=0).stop 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 msgr2=0x55ae0909eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 --2- 172.21.15.191:0/3262643690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db7400 0x55ae0909eb00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55ae09c34480 tx=0x55ae09108660 comp rx=0 tx=0).stop 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 reap_dead start 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b0e18700 1 -- 172.21.15.191:0/3262643690 reap_dead start 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 shutdown_connections 2024-04-27T04:35:48.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 >> 172.21.15.191:0/3262643690 conn(0x55ae08db6c00 msgr2=0x55ae08ea3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 shutdown_connections 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/3262643690 wait complete. 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.924+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae092a8000 msgr2=0x55ae090a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae092a8000 0x55ae090a1700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ae091afd70 tx=0x55ae091fc5a0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 msgr2=0x55ae090a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55ae09228210 tx=0x55ae091fc8a0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 shutdown_connections 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae092a8000 0x55ae090a1700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/553937887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae09217400 0x55ae090a0680 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 >> 172.21.15.191:0/553937887 conn(0x55ae09217000 msgr2=0x55ae08ea4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 shutdown_connections 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/553937887 wait complete. 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150000 msgr2=0x55ae0909f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150000 0x55ae0909f080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55ae09c34840 tx=0x55ae09108240 comp rx=0 tx=0).stop 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 msgr2=0x55ae0909e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x55ae09b8fad0 tx=0x55ae08ea7a40 comp rx=0 tx=0).stop 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 shutdown_connections 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ae09150000 0x55ae0909f080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 --2- 172.21.15.191:0/4283159287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae08db6800 0x55ae0909e580 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 >> 172.21.15.191:0/4283159287 conn(0x55ae08db6400 msgr2=0x55ae08ea3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 shutdown_connections 2024-04-27T04:35:48.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.925+0000 7f92b366fa80 1 -- 172.21.15.191:0/4283159287 wait complete. 2024-04-27T04:35:48.942 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:48.943 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-27T04:35:48.947 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-27T04:35:48.947 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-04-27T04:35:48.947 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-27T04:35:48.947 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-27T04:35:48.947 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-04-27T04:35:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:48.993 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.992+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.992+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.992+0000 7f4e5030ca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ec3f74ec00 0x55ec3fa36580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:48.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.992+0000 7f4e5030ca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ec3fa79500 con 0x55ec3f74e800 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.992+0000 7f4e5030ca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ec3f7f0a00 con 0x55ec3f74ec00 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4dab5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ec3f74ec00 0x55ec3fa36580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4dab5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55ec3f74ec00 0x55ec3fa36580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49402/0 (socket says 172.21.15.191:49402) 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4dab5700 1 -- 172.21.15.191:0/3323872215 learned_addr learned my addr 172.21.15.191:0/3323872215 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4dab5700 1 -- 172.21.15.191:0/3323872215 >> v1:172.21.15.138:6789/0 conn(0x55ec3f74e800 legacy=0x55ec3f83a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4dab5700 1 -- 172.21.15.191:0/3323872215 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f777d40 con 0x55ec3f74ec00 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/3323872215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74ec00 0x55ec3fa36580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55ec3fa31890 tx=0x55ec3f83f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13affd733f91b3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4d2b4700 1 -- 172.21.15.191:0/3323872215 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f8074a0 con 0x55ec3f74ec00 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4d2b4700 1 -- 172.21.15.191:0/3323872215 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3fa79800 con 0x55ec3f74ec00 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.993+0000 7f4e4d2b4700 1 -- 172.21.15.191:0/3323872215 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f807600 con 0x55ec3f74ec00 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3323872215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74ec00 msgr2=0x55ec3fa36580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/3323872215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74ec00 0x55ec3fa36580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55ec3fa31890 tx=0x55ec3f83f920 comp rx=0 tx=0).stop 2024-04-27T04:35:48.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3323872215 shutdown_connections 2024-04-27T04:35:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/3323872215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74ec00 0x55ec3fa36580 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3323872215 >> 172.21.15.191:0/3323872215 conn(0x55ec3f74e400 msgr2=0x55ec3f83a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3323872215 shutdown_connections 2024-04-27T04:35:48.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3323872215 wait complete. 2024-04-27T04:35:49.012 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.994+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e5030ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e5030ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3f7f0500 con 0x55ec3f74e800 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4e2b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4e2b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49404/0 (socket says 172.21.15.191:49404) 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4091710498 learned_addr learned my addr 172.21.15.191:0/4091710498 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4091710498 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f777ba0 con 0x55ec3f74e800 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4091710498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55ec40515890 tx=0x55ec3f83fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec36d0dc54e0e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4cab3700 1 -- 172.21.15.191:0/4091710498 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f8071e0 con 0x55ec3f74e800 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4cab3700 1 -- 172.21.15.191:0/4091710498 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3fa79b00 con 0x55ec3f74e800 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.995+0000 7f4e4cab3700 1 -- 172.21.15.191:0/4091710498 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f8074a0 con 0x55ec3f74e800 2024-04-27T04:35:49.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4091710498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 msgr2=0x55ec3fa36580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4091710498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55ec40515890 tx=0x55ec3f83fb60 comp rx=0 tx=0).stop 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4091710498 shutdown_connections 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4091710498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4091710498 >> 172.21.15.191:0/4091710498 conn(0x55ec3f74e400 msgr2=0x55ec3f83b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4091710498 shutdown_connections 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4091710498 wait complete. 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.996+0000 7f4e5030ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3f7f03c0 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4e2b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4e2b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49414/0 (socket says 172.21.15.191:49414) 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4087249563 learned_addr learned my addr 172.21.15.191:0/4087249563 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4087249563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f777a00 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55ec405156b0 tx=0x55ec3f83fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4b2b0700 1 -- 172.21.15.191:0/4087249563 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f807600 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4b2b0700 1 -- 172.21.15.191:0/4087249563 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3f7efc80 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec3f777860 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e4b2b0700 1 -- 172.21.15.191:0/4087249563 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f8071e0 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.997+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec3f7776c0 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4b2b0700 1 -- 172.21.15.191:0/4087249563 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ec404f0b00 con 0x55ec3f74e800 2024-04-27T04:35:49.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e5030ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4b2b0700 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0000 0x55ec3fa37080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e5030ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3f78a640 con 0x55ec3f74f400 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4b2b0700 1 -- 172.21.15.191:0/4087249563 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ec3f7c9180 con 0x55ec3f74e800 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0000 0x55ec3fa37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4eab7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4eab7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49422/0 (socket says 172.21.15.191:49422) 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4eab7700 1 -- 172.21.15.191:0/3597696094 learned_addr learned my addr 172.21.15.191:0/3597696094 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0000 0x55ec3fa37080 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55ec405cd020 tx=0x55ec3fa9a240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.998+0000 7f4e4eab7700 1 -- 172.21.15.191:0/3597696094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f776000 con 0x55ec3f74f400 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/3597696094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x55ec405cc3c0 tx=0x55ec3fa9b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94726367f1f157e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e48aab700 1 -- 172.21.15.191:0/3597696094 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f807600 con 0x55ec3f74f400 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e48aab700 1 -- 172.21.15.191:0/3597696094 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3fa78300 con 0x55ec3f74f400 2024-04-27T04:35:49.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e48aab700 1 -- 172.21.15.191:0/3597696094 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f8071e0 con 0x55ec3f74f400 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3597696094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 msgr2=0x55ec3fa36b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/3597696094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x55ec405cc3c0 tx=0x55ec3fa9b0e0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3597696094 shutdown_connections 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/3597696094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3597696094 >> 172.21.15.191:0/3597696094 conn(0x55ec3f74ec00 msgr2=0x55ec3f83b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3597696094 shutdown_connections 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 -- 172.21.15.191:0/3597696094 wait complete. 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:48.999+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e5030ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e5030ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3f78a8c0 con 0x55ec3f74f400 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e4eab7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e4eab7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49430/0 (socket says 172.21.15.191:49430) 2024-04-27T04:35:49.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 learned_addr learned my addr 172.21.15.191:0/4246531432 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f776ea0 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55ec405cc210 tx=0x55ec3fa9acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e472a8700 1 -- 172.21.15.191:0/4246531432 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f807a20 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e472a8700 1 -- 172.21.15.191:0/4246531432 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3f7efc80 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e472a8700 1 -- 172.21.15.191:0/4246531432 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f807600 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec3f776820 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.000+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec3f7769c0 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e472a8700 1 -- 172.21.15.191:0/4246531432 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ec404f0dc0 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e472a8700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0c00 0x55ec3fa37600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e472a8700 1 -- 172.21.15.191:0/4246531432 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ec3f7c9340 con 0x55ec3f74f400 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0c00 0x55ec3fa37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3f74f800 0x55ec3fa37b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.001+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3f74f800 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3f74f800 0x55ec3fa37b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0c00 0x55ec3fa37600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ec3fa31890 tx=0x55ec3f83f800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3f74f800 0x55ec3fa37b80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 20 realm 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1000 con 0x55ec3f74f800 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:49.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.002+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1400 con 0x55ec3f74f800 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1800 con 0x55ec3f74f800 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fae1c00 0x55ec3fa38100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1000 con 0x55ec3fae1c00 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fae1c00 0x55ec3fa38100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fae1c00 0x55ec3fa38100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.003+0000 7f4e5030ca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e5030ca80 20 searching for the correct realm 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1400 con 0x55ec3fae1c00 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fae1400 0x55ec3fa38c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1000 con 0x55ec3fae1400 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fae1400 0x55ec3fa38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fae1400 0x55ec3fa38c00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fae1000 0x55ec3fa39180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.004+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.005+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fae1000 0x55ec3fa39180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.005+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fae1000 0x55ec3fa39180 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.005+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.005+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.005+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.005+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1400 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.006+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1400 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.007+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.023 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.008+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.009+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1400 2024-04-27T04:35:49.024 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.010+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1400 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1400 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1c00 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.011+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1400 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:49.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3f74f800 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.012+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3f74f800 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6800 con 0x55ec3fae1c00 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:49.040 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 zone default found 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 4 Realm: () 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 zonegroup default 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3fae1000 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7000 con 0x55ec3fae1c00 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7400 con 0x55ec3fae1c00 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.013+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.014+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.014+0000 7f4e5030ca80 20 started sync module instance, tier type = 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.014+0000 7f4e5030ca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.014+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fae1400 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.015+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.015+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94473168280320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6800 con 0x55ec3fae1400 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.015+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7000 con 0x55ec3f74f800 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.016+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94473168280320] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.016+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.016+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94473168281472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7400 con 0x55ec3f74f800 2024-04-27T04:35:49.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.016+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3fae1c00 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.017+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94473168281472] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.017+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.017+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94473168282624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3fae1c00 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.017+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3fae1400 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.018+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.018+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94473168283776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fae1800 con 0x55ec3fae1400 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.018+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3f74f800 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.018+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94473168282624] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.019+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94473168283776] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.020+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.020+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94473168284928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3f74f800 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.020+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6800 con 0x55ec3fae1000 2024-04-27T04:35:49.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.021+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.021+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94473168286080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7c00 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.021+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7000 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.021+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94473168284928] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.022+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94473168286080] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.022+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.022+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94473168287232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7400 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.022+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6000 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94473168287232] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94473168288384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fae1000 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=0 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=1 2024-04-27T04:35:49.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=2 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=3 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=4 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=5 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.023+0000 7f4e5030ca80 20 add_watcher() i=6 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.024+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94473168288384] v33'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.024+0000 7f4e5030ca80 20 add_watcher() i=7 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.024+0000 7f4e5030ca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.024+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6800 con 0x55ec3f74f800 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.024+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.024+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94473169879040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7c00 con 0x55ec3f74f800 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.025+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94473169879040] v33'224 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.025+0000 7f4e35284700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.025+0000 7f4e5030ca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:49.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.025+0000 7f4e5030ca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.025+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.025+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e5030ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e5030ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3f7f03c0 con 0x55ec3fba7400 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e4dab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e4dab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49438/0 (socket says 172.21.15.191:49438) 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e4dab5700 1 -- 172.21.15.191:0/1858162945 learned_addr learned my addr 172.21.15.191:0/1858162945 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e4dab5700 1 -- 172.21.15.191:0/1858162945 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f776b60 con 0x55ec3fba7400 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/1858162945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55ec3fb3b860 tx=0x55ec3f83f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a57f0bd213963 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e34a83700 1 -- 172.21.15.191:0/1858162945 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3f807760 con 0x55ec3fba7400 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e34a83700 1 -- 172.21.15.191:0/1858162945 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3fbc8000 con 0x55ec3fba7400 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.026+0000 7f4e34a83700 1 -- 172.21.15.191:0/1858162945 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3fbca000 con 0x55ec3fba7400 2024-04-27T04:35:49.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- 172.21.15.191:0/1858162945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 msgr2=0x55ec3fa38680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/1858162945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55ec3fb3b860 tx=0x55ec3f83f980 comp rx=0 tx=0).stop 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- 172.21.15.191:0/1858162945 shutdown_connections 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/1858162945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- 172.21.15.191:0/1858162945 >> 172.21.15.191:0/1858162945 conn(0x55ec3fba7000 msgr2=0x55ec3f83c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- 172.21.15.191:0/1858162945 shutdown_connections 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- 172.21.15.191:0/1858162945 wait complete. 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 Processor -- start 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- start start 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.027+0000 7f4e5030ca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3f7f0500 con 0x55ec3fba7400 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e4dab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e4dab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49448/0 (socket says 172.21.15.191:49448) 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 learned_addr learned my addr 172.21.15.191:0/411336721 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3f776d00 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55ec3fbb8810 tx=0x55ec3fb888a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e33280700 1 -- 172.21.15.191:0/411336721 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3fbca160 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e33280700 1 -- 172.21.15.191:0/411336721 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec3fbc8180 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec3f776340 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e33280700 1 -- 172.21.15.191:0/411336721 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ec3fbca2c0 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.028+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec3f777d40 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e33280700 1 -- 172.21.15.191:0/411336721 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ec404f1b80 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e33280700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fc46000 0x55ec3fa39700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e33280700 1 -- 172.21.15.191:0/411336721 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ec3f7c9500 con 0x55ec3fba7400 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fc46000 0x55ec3fa39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3fba6000 0x55ec3fc5e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fba6000 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3fba6000 0x55ec3fc5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.029+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fc46000 0x55ec3fa39700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55ec3fb3b890 tx=0x55ec3fb885a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.030+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3fba6000 0x55ec3fc5e000 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.031+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'59 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fba6000 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.031+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fba6800 0x55ec3fc5e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.031+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc46400 con 0x55ec3fba6800 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.031+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fba6800 0x55ec3fc5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.031+0000 7f4e4dab5700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fba6800 0x55ec3fc5e580 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.032+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.032+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fc46800 0x55ec3fc5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.032+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc46c00 con 0x55ec3fc46800 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.032+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fc46800 0x55ec3fc5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.032+0000 7f4e4eab7700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fc46800 0x55ec3fc5eb00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.033+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.033+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47000 con 0x55ec3fba6000 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.034+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'225 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fba6000 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.034+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fc47400 0x55ec3fc5f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.034+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47800 con 0x55ec3fc47400 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.034+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fc47400 0x55ec3fc5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.035+0000 7f4e4e2b6700 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fc47400 0x55ec3fc5f080 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.035+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'77 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.036+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47c00 con 0x55ec3fc46800 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.036+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'58 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.036+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47000 con 0x55ec3fba6000 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.038+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fba6000 2024-04-27T04:35:49.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.038+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3fba6800 2024-04-27T04:35:49.053 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.039+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'53 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.039+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3fc47400 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.040+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.040+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fc46800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.041+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.041+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c000 con 0x55ec3fc47400 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.042+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'78 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.042+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c400 con 0x55ec3fba6800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.043+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'32 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.043+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c800 con 0x55ec3fc47400 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.044+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'38 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.044+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8cc00 con 0x55ec3fba6800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.045+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'68 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.054 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.045+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d000 con 0x55ec3fba6000 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.046+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'226 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fba6000 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.046+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d400 con 0x55ec3fc46800 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.047+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'68 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.047+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d800 con 0x55ec3fc47400 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.048+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'79 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.048+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8dc00 con 0x55ec3fc46800 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.049+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'38 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.049+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c000 con 0x55ec3fba6000 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.050+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fba6000 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.050+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c800 con 0x55ec3fba6000 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.051+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'227 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fba6000 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.051+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c400 con 0x55ec3fc46800 2024-04-27T04:35:49.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.052+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'69 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.052+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3fc47400 2024-04-27T04:35:49.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.053+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'42 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.053+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3fc46800 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.054+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.054+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47800 con 0x55ec3fc46800 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.055+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'44 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.055+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47000 con 0x55ec3fc46800 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.056+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.056+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc46400 con 0x55ec3fc46800 2024-04-27T04:35:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.056+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'59 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.057+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3fba6800 2024-04-27T04:35:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.058+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.059 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.058+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d400 con 0x55ec3fc46800 2024-04-27T04:35:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.058+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'38 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.058+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8dc00 con 0x55ec3fba6800 2024-04-27T04:35:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.059+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'54 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.059+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c400 con 0x55ec3fc46800 2024-04-27T04:35:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.060+0000 7f4e4eab7700 1 -- 172.21.15.191:0/411336721 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'70 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fc46800 2024-04-27T04:35:49.062 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.060+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7c00 con 0x55ec3fba6800 2024-04-27T04:35:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.061+0000 7f4e4dab5700 1 -- 172.21.15.191:0/411336721 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'69 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fba6800 2024-04-27T04:35:49.063 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.061+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fc47400 2024-04-27T04:35:49.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.063+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.063+0000 7f4e30a7b700 20 reqs_thread_entry: start 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e2ea77700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:19 2024 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e2e255700 20 reqs_thread_entry: start 2024-04-27T04:35:49.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47c00 con 0x55ec3fae1400 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7c00 con 0x55ec3f74f800 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.064+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 bud.buckets [call out=184b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 : miss 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 8.9 8:91bc09d5:root::.bucket.meta.myfoo6%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3fae1400 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(64 .bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=212b] v0'0 uv45 ondisk = 0) v8 ==== 333+0+629 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 20 rados_obj.operate() r=0 bl.length=353 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x17 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 4.f 4:f395ffb4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.065+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 4.36 4:6f3c98dd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8cc00 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.066+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.066+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 4.38 4:1f62a13f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8dc00 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.066+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.066+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 4.28 4:153413f6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d000 con 0x55ec3f74f800 2024-04-27T04:35:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.066+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.066+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 4.3f 4:fd62470d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d800 con 0x55ec3fae1400 2024-04-27T04:35:49.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.067+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.067+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 4.26 4:677ab003:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c000 con 0x55ec3fae1000 2024-04-27T04:35:49.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.067+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.067+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 4.21 4:869f5d7c:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c800 con 0x55ec3fae1000 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 4.38 4:1c9a7278:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47c00 con 0x55ec3f74f800 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 4.12 4:4afd3883:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3f74f800 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3fae1000 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [call out=289b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+289 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.068+0000 7f4e5030ca80 10 cls_bucket_list_unordered: got thirteen 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.1d 4:ba7e5be2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7c00 con 0x55ec3fae1c00 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 20 get_obj_state: rctx=0x55ec3fb4de30 obj=myfoo6:thirteen state=0x55ec3fd0e1e8 s->prefetch_data=0 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:76 3.2fs0 3:f5593e88:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fae1c00 2024-04-27T04:35:49.074 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(76 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen [getxattrs out=819b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 238+0+835 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 10 manifest: total_size = 8 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 20 get_obj_state: setting s->obj_tag to _UISvwJ96uF1DptECUU3t71hN8P7WEkk 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 20 get_obj_state: rctx=0x55ec3fb4de30 obj=myfoo6:thirteen state=0x55ec3fd0e1e8 s->prefetch_data=0 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 20 prepare_atomic_modification: state is not atomic. state=0x55ec3fd0e1e8 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.069+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d400 con 0x55ec3fae1000 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.071+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(77 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [stat,call,call] v33'9 uv9 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.071+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:78 3.2fs0 3:f5593e88:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c400 con 0x55ec3fae1c00 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(78 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7_thirteen [call] v33'3 uv3 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:79 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47000 con 0x55ec3fae1000 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 20 list_objects_ordered: starting attempt 1 2024-04-27T04:35:49.075 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 4.f 4:f395ffb4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc46400 con 0x55ec3f74f800 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 4.36 4:6f3c98dd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8cc00 con 0x55ec3f74f800 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 4.38 4:1f62a13f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8dc00 con 0x55ec3f74f800 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:83 4.28 4:153413f6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d000 con 0x55ec3f74f800 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:84 4.3f 4:fd62470d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c000 con 0x55ec3fae1400 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 4.26 4:677ab003:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c800 con 0x55ec3fae1000 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 4.21 4:869f5d7c:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d400 con 0x55ec3fae1000 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:87 4.38 4:1c9a7278:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3f74f800 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 4.12 4:4afd3883:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3f74f800 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:89 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47c00 con 0x55ec3fae1000 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 4.1d 4:ba7e5be2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92000 con 0x55ec3fae1c00 2024-04-27T04:35:49.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(86 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(82 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.072+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(80 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(83 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(81 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(87 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(88 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(85 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(84 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(90 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(79 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [stat,call,call] v33'10 uv10 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.073+0000 7f4e482aa700 20 handle_completion(): completion ok for obj=thirteen 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(89 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T04:35:49.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7])"; uploads.size()=0; is_truncated=0 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]) 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:91 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92400 con 0x55ec3f74f800 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 34 ==== osd_op_reply(91 myfoo6 [call out=48b,stat out=16b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 276+0+305 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 lifecycle: adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:49.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.074+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:92 8.9 8:91bc09d5:root::.bucket.meta.myfoo6%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92c00 con 0x55ec3fae1400 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(92 .bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v33'47 uv47 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e5030ca80 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x17 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e5030ca80 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e5030ca80 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e5030ca80 10 lifecycle: distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7, ofs0, ns] 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:93 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94473169882496 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93400 con 0x55ec3fae1000 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== watch-notify(notify (1) cookie 94473168287232 notify 141733920801 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x55ec3fc11ba0 con 0x55ec3fae1000 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.075+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(93 notify.6 [notify cookie 94473169882496 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e482aa700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920801 cookie 94473168287232 notifier 4820 bl.length()=721 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e482aa700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x17 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e482aa700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e482aa700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T04:35:49.082 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e482aa700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:94 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47800 con 0x55ec3fae1000 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(94 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== watch-notify(notify_complete (2) cookie 94473169882496 notify 141733920801 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ec3fc11d40 con 0x55ec3fae1000 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e5030ca80 20 lifecycle: handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): orig_sources= new_sources= 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e5030ca80 20 lifecycle: handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): potential sources added=[] removed=[] 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e5030ca80 20 lifecycle: handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): orig_dests= new_dests= 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e5030ca80 20 lifecycle: handle_bi_update(): bucket=:myfoo6[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7]): potential dests added=[] removed=[] 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.076+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 6.14 6:2f3f5e88:lc::lc.17:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93800 con 0x55ec3fc47400 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.077+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(33 lc.17 [call] v33'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.083 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.077+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:34 6.14 6:2f3f5e88:lc::lc.17:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93c00 con 0x55ec3fc47400 2024-04-27T04:35:49.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.078+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(34 lc.17 [call] v33'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.078+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:35 6.14 6:2f3f5e88:lc::lc.17:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47000 con 0x55ec3fc47400 2024-04-27T04:35:49.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(35 lc.17 [call] v33'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fc47400 2024-04-27T04:35:49.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:95 4.f 4:f395ffb4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc46400 con 0x55ec3f74f800 2024-04-27T04:35:49.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 4.36 4:6f3c98dd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47c00 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:97 4.38 4:1f62a13f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:98 4.28 4:153413f6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:99 4.3f 4:fd62470d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6400 con 0x55ec3fae1400 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 4.26 4:677ab003:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d800 con 0x55ec3fae1000 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 4.21 4:869f5d7c:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c000 con 0x55ec3fae1000 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:102 4.38 4:1c9a7278:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8cc00 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:103 4.12 4:4afd3883:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8dc00 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:104 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d000 con 0x55ec3fae1000 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:105 4.1d 4:ba7e5be2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c800 con 0x55ec3fae1c00 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(105 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 35 ==== osd_op_reply(95 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 36 ==== osd_op_reply(97 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 37 ==== osd_op_reply(98 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 38 ==== osd_op_reply(102 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(99 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 39 ==== osd_op_reply(96 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(100 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== osd_op_reply(101 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 40 ==== osd_op_reply(103 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== osd_op_reply(104 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.079+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:106 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d400 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.080+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 41 ==== osd_op_reply(106 bud.buckets [call] v33'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.080+0000 7f4e5030ca80 10 cache get: name=default.rgw.meta+root+myfoo6 : hit (requested=0x11, cached=0x17) 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.080+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:107 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93000 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.081+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 42 ==== osd_op_reply(107 myfoo6 [call,call,delete] v33'19 uv19 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.081+0000 7f4e5030ca80 10 removing default.rgw.meta+root+myfoo6 from cache 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.081+0000 7f4e5030ca80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.081+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 94473169882496 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92800 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== watch-notify(notify (1) cookie 94473168283776 notify 141733920799 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x55ec3fc0fa00 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e482aa700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920799 cookie 94473168283776 notifier 4820 bl.length()=150 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(108 notify.3 [notify cookie 94473169882496 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e482aa700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3f74fc00 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== watch-notify(notify_complete (2) cookie 94473169882496 notify 141733920799 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ec3fc0d040 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.082+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:110 8.9 8:91bc09d5:root::.bucket.meta.myfoo6%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92c00 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.083+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(110 .bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [call,call,delete] v33'48 uv48 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.083+0000 7f4e5030ca80 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 from cache 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.083+0000 7f4e5030ca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7, ofs0, ns] 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.083+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94473169882496 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47800 con 0x55ec3fae1000 2024-04-27T04:35:49.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.083+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== watch-notify(notify (1) cookie 94473168287232 notify 141733920802 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x55ec3fc11d40 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e48aab700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920802 cookie 94473168287232 notifier 4820 bl.length()=207 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(111 notify.6 [notify cookie 94473169882496 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e48aab700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1a000 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 33 ==== watch-notify(notify_complete (2) cookie 94473169882496 notify 141733920802 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ec3fc10340 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 34 ==== osd_op_reply(112 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:113 4.f 4:f395ffb4:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc46400 con 0x55ec3f74f800 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:114 4.36 4:6f3c98dd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47c00 con 0x55ec3f74f800 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:115 4.38 4:1f62a13f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47000 con 0x55ec3f74f800 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:116 4.28 4:153413f6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92400 con 0x55ec3f74f800 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:117 4.3f 4:fd62470d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93400 con 0x55ec3fae1400 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:118 4.26 4:677ab003:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92000 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:119 4.21 4:869f5d7c:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93800 con 0x55ec3fae1000 2024-04-27T04:35:49.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:120 4.38 4:1c9a7278:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93c00 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:121 4.12 4:4afd3883:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8d800 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:122 4.26 4:662937c7:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c000 con 0x55ec3fae1000 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.084+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:123 4.1d 4:ba7e5be2:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8cc00 con 0x55ec3fae1c00 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.085+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 35 ==== osd_op_reply(119 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.6 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 43 ==== osd_op_reply(114 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.1 [delete] v33'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 44 ==== osd_op_reply(113 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.0 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 36 ==== osd_op_reply(118 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.5 [delete] v33'11 uv11 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 45 ==== osd_op_reply(121 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.8 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(117 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.4 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 46 ==== osd_op_reply(116 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.3 [delete] v33'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 47 ==== osd_op_reply(115 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.2 [delete] v33'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(123 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.10 [delete] v33'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.086+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 37 ==== osd_op_reply(122 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.9 [delete] v33'12 uv12 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 48 ==== osd_op_reply(120 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7.7 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 : miss 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:124 6.1b 6:db1d084b:::pubsub..bucket.myfoo6%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8dc00 con 0x55ec3fae1c00 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(124 pubsub..bucket.myfoo6/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 info.flags=0x0 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 10 adding default.rgw.log++pubsub..bucket.myfoo6/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 to cache LRU end 2024-04-27T04:35:49.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.087+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:125 6.1b 6:db1d084b:::pubsub..bucket.myfoo6%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc8c400 con 0x55ec3fae1c00 2024-04-27T04:35:49.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.088+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(125 pubsub..bucket.myfoo6/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.7 [delete] v33'29 uv23 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.088+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:126 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba6c00 con 0x55ec3f74f800 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.089+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 49 ==== osd_op_reply(126 bud.buckets [call] v33'5 uv5 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.089+0000 7f4e5030ca80 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.089+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:127 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7c00 con 0x55ec3f74f800 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 50 ==== osd_op_reply(127 NX5QOQKC6BH2IDN8HC7A [delete] v33'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e5030ca80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:128 7.3 7:c609908c:::notify.5:head [notify cookie 94473169882496 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fba7800 con 0x55ec3fae1000 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 38 ==== watch-notify(notify (1) cookie 94473168286080 notify 141733920803 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55ec3fc10340 con 0x55ec3fae1000 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 39 ==== osd_op_reply(128 notify.5 [notify cookie 94473169882496 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e48aab700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920803 cookie 94473168286080 notifier 4820 bl.length()=170 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.090+0000 7f4e48aab700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:129 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1a400 con 0x55ec3fae1000 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.091+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 40 ==== osd_op_reply(129 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.091+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 41 ==== watch-notify(notify_complete (2) cookie 94473169882496 notify 141733920803 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ec3fc104e0 con 0x55ec3fae1000 2024-04-27T04:35:49.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.091+0000 7f4e5030ca80 10 removing email index: 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.091+0000 7f4e5030ca80 10 removing user buckets index 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.091+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:130 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1a000 con 0x55ec3f74f800 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 51 ==== osd_op_reply(130 bud.buckets [delete] v33'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e5030ca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:131 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94473169882496 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1a800 con 0x55ec3fae1000 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 42 ==== watch-notify(notify (1) cookie 94473168287232 notify 141733920804 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55ec3fc104e0 con 0x55ec3fae1000 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e48aab700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920804 cookie 94473168287232 notifier 4820 bl.length()=160 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 43 ==== osd_op_reply(131 notify.6 [notify cookie 94473169882496 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e48aab700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:132 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd24000 con 0x55ec3fae1000 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 44 ==== osd_op_reply(132 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 45 ==== watch-notify(notify_complete (2) cookie 94473169882496 notify 141733920804 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ec3fc10680 con 0x55ec3fae1000 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.092+0000 7f4e5030ca80 10 removing user index: bud 2024-04-27T04:35:49.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.093+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:133 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1ac00 con 0x55ec3fae1400 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.093+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(133 bud [call,call,delete] v33'3 uv3 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.093+0000 7f4e5030ca80 10 removing default.rgw.meta+users.uid+bud from cache 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.093+0000 7f4e5030ca80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.093+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94473169882496 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1b000 con 0x55ec3fae1000 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.094+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 46 ==== watch-notify(notify (1) cookie 94473168286080 notify 141733920805 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55ec3fc10680 con 0x55ec3fae1000 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.094+0000 7f4e482aa700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920805 cookie 94473168286080 notifier 4820 bl.length()=152 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.094+0000 7f4e482aa700 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd24c00 con 0x55ec3fae1000 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.094+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 47 ==== osd_op_reply(134 notify.5 [notify cookie 94473169882496 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.094+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 48 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.094+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 49 ==== watch-notify(notify_complete (2) cookie 94473169882496 notify 141733920805 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ec3fc10820 con 0x55ec3fae1000 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.095+0000 7f4e35284700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.095+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:136 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94473169879040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1b400 con 0x55ec3f74f800 2024-04-27T04:35:49.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.096+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 52 ==== osd_op_reply(136 data_loggenerations_metadata [watch unwatch cookie 94473169879040] v33'228 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.096+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:137 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94473168280320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1b800 con 0x55ec3fae1400 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.097+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(137 notify.0 [watch unwatch cookie 94473168280320] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.097+0000 7f4e5030ca80 20 remove_watcher() i=0 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.097+0000 7f4e5030ca80 2 removed watcher, disabling cache 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.097+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:138 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94473168281472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd1bc00 con 0x55ec3f74f800 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.098+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 53 ==== osd_op_reply(138 notify.1 [watch unwatch cookie 94473168281472] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.098+0000 7f4e5030ca80 20 remove_watcher() i=1 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.098+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:139 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94473168282624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93000 con 0x55ec3fae1c00 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.099+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/4246531432 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(139 notify.2 [watch unwatch cookie 94473168282624] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b200 con 0x55ec3fae1c00 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.099+0000 7f4e5030ca80 20 remove_watcher() i=2 2024-04-27T04:35:49.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.099+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:140 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94473168283776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92400 con 0x55ec3fae1400 2024-04-27T04:35:49.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.100+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(140 notify.3 [watch unwatch cookie 94473168283776] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9b440 con 0x55ec3fae1400 2024-04-27T04:35:49.109 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.100+0000 7f4e5030ca80 20 remove_watcher() i=3 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.100+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94473168284928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd92000 con 0x55ec3f74f800 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.101+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.1 v2:172.21.15.191:6808/41886 54 ==== osd_op_reply(141 notify.4 [watch unwatch cookie 94473168284928] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3f74f800 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.101+0000 7f4e5030ca80 20 remove_watcher() i=4 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.101+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94473168286080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93800 con 0x55ec3fae1000 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.102+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 50 ==== osd_op_reply(142 notify.5 [watch unwatch cookie 94473168286080] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.102+0000 7f4e5030ca80 20 remove_watcher() i=5 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.102+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:143 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94473168287232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fd93c00 con 0x55ec3fae1000 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.103+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 51 ==== osd_op_reply(143 notify.6 [watch unwatch cookie 94473168287232] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.103+0000 7f4e5030ca80 20 remove_watcher() i=6 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.103+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:144 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94473168288384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ec3fc47800 con 0x55ec3fae1000 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e4eab7700 1 -- 172.21.15.191:0/4246531432 <== osd.0 v2:172.21.15.138:6800/39496 52 ==== osd_op_reply(144 notify.7 [watch unwatch cookie 94473168288384] v33'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ec3fb9ab40 con 0x55ec3fae1000 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 20 remove_watcher() i=7 2024-04-27T04:35:49.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fae1000 msgr2=0x55ec3fa39180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fae1000 0x55ec3fa39180 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3f74f800 msgr2=0x55ec3fa37b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3f74f800 0x55ec3fa37b80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fae1400 msgr2=0x55ec3fa38c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fae1400 0x55ec3fa38c00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fae1c00 msgr2=0x55ec3fa38100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fae1c00 0x55ec3fa38100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0c00 msgr2=0x55ec3fa37600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0c00 0x55ec3fa37600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ec3fa31890 tx=0x55ec3f83f800 comp rx=0 tx=0).stop 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 msgr2=0x55ec3fa36b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4246531432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74f400 0x55ec3fa36b00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55ec405cc210 tx=0x55ec3fa9acc0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 reap_dead start 2024-04-27T04:35:49.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e4dab5700 1 -- 172.21.15.191:0/4246531432 reap_dead start 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 shutdown_connections 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 >> 172.21.15.191:0/4246531432 conn(0x55ec3f74ec00 msgr2=0x55ec3f83b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 shutdown_connections 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.104+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4246531432 wait complete. 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fba6800 msgr2=0x55ec3fc5e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55ec3fba6800 0x55ec3fc5e580 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3fba6000 msgr2=0x55ec3fc5e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55ec3fba6000 0x55ec3fc5e000 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fc47400 msgr2=0x55ec3fc5f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55ec3fc47400 0x55ec3fc5f080 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fc46800 msgr2=0x55ec3fc5eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55ec3fc46800 0x55ec3fc5eb00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fc46000 msgr2=0x55ec3fa39700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fc46000 0x55ec3fa39700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55ec3fb3b890 tx=0x55ec3fb885a0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 msgr2=0x55ec3fa38680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/411336721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3fba7400 0x55ec3fa38680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55ec3fbb8810 tx=0x55ec3fb888a0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e4e2b6700 1 -- 172.21.15.191:0/411336721 reap_dead start 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 shutdown_connections 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 >> 172.21.15.191:0/411336721 conn(0x55ec3fba7000 msgr2=0x55ec3f83c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 shutdown_connections 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/411336721 wait complete. 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0000 msgr2=0x55ec3fa37080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0000 0x55ec3fa37080 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55ec405cd020 tx=0x55ec3fa9a240 comp rx=0 tx=0).stop 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 msgr2=0x55ec3fa36580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55ec405156b0 tx=0x55ec3f83fa40 comp rx=0 tx=0).stop 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 shutdown_connections 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55ec3fae0000 0x55ec3fa37080 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.113 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 --2- 172.21.15.191:0/4087249563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec3f74e800 0x55ec3fa36580 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 >> 172.21.15.191:0/4087249563 conn(0x55ec3f74e400 msgr2=0x55ec3f83b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 shutdown_connections 2024-04-27T04:35:49.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.105+0000 7f4e5030ca80 1 -- 172.21.15.191:0/4087249563 wait complete. 2024-04-27T04:35:49.114 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-27T04:35:49.114 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-27T04:35:49.114 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-04-27T04:35:49.160 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:35:49.161 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.159+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.159+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2aca9fca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556732f06800 0x5567331ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2aca9fca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556733231500 con 0x556732f06c00 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2aca9fca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x556732fa8a00 con 0x556732f06800 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac89a6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556732f06800 0x5567331ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.178 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac81a5700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x556732f06c00 0x556732ff2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:48010/0 (socket says 172.21.15.191:48010) 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac89a6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556732f06800 0x5567331ee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49458/0 (socket says 172.21.15.191:49458) 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac81a5700 1 -- 172.21.15.191:0/3845572610 learned_addr learned my addr 172.21.15.191:0/3845572610 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac89a6700 1 -- 172.21.15.191:0/3845572610 >> v1:172.21.15.138:6789/0 conn(0x556732f06c00 legacy=0x556732ff2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac89a6700 1 -- 172.21.15.191:0/3845572610 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2fd40 con 0x556732f06800 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/3845572610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x5567331e99e0 tx=0x556732ff7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ba22b4e5930f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.160+0000 7f2ac79a4700 1 -- 172.21.15.191:0/3845572610 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf4a0 con 0x556732f06800 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2ac79a4700 1 -- 172.21.15.191:0/3845572610 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556733231800 con 0x556732f06800 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2ac79a4700 1 -- 172.21.15.191:0/3845572610 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf600 con 0x556732f06800 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3845572610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 msgr2=0x5567331ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/3845572610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x5567331e99e0 tx=0x556732ff7920 comp rx=0 tx=0).stop 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3845572610 shutdown_connections 2024-04-27T04:35:49.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/3845572610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3845572610 >> 172.21.15.191:0/3845572610 conn(0x556732f06400 msgr2=0x556732ff2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3845572610 shutdown_connections 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.161+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3845572610 wait complete. 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2aca9fca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2aca9fca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556732fa8500 con 0x556732f06800 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2ac89a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2ac89a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49468/0 (socket says 172.21.15.191:49468) 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2ac89a6700 1 -- 172.21.15.191:0/3729382157 learned_addr learned my addr 172.21.15.191:0/3729382157 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.162+0000 7f2ac89a6700 1 -- 172.21.15.191:0/3729382157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2fba0 con 0x556732f06800 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/3729382157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x556733cde0c0 tx=0x556732ff7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c0227e7d327204 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2ac71a3700 1 -- 172.21.15.191:0/3729382157 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf760 con 0x556732f06800 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2ac71a3700 1 -- 172.21.15.191:0/3729382157 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556733231b00 con 0x556732f06800 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2ac71a3700 1 -- 172.21.15.191:0/3729382157 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf4a0 con 0x556732f06800 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3729382157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 msgr2=0x5567331ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/3729382157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x556733cde0c0 tx=0x556732ff7800 comp rx=0 tx=0).stop 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3729382157 shutdown_connections 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/3729382157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3729382157 >> 172.21.15.191:0/3729382157 conn(0x556732f06400 msgr2=0x556732ff3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3729382157 shutdown_connections 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3729382157 wait complete. 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.163+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.181 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2aca9fca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2aca9fca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556732fa83c0 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac89a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac89a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49474/0 (socket says 172.21.15.191:49474) 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac89a6700 1 -- 172.21.15.191:0/102259287 learned_addr learned my addr 172.21.15.191:0/102259287 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac89a6700 1 -- 172.21.15.191:0/102259287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2fa00 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x556733cdf980 tx=0x556732ff7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac59a0700 1 -- 172.21.15.191:0/102259287 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf600 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac59a0700 1 -- 172.21.15.191:0/102259287 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556733231c80 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.164+0000 7f2ac59a0700 1 -- 172.21.15.191:0/102259287 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf8c0 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556732f2f860 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556732f2f6c0 con 0x556732f06800 2024-04-27T04:35:49.182 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2aca9fca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2aca9fca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556732f42640 con 0x556732f07400 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2ac91a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2ac59a0700 1 -- 172.21.15.191:0/102259287 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556733ca8b00 con 0x556732f06800 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2ac91a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49476/0 (socket says 172.21.15.191:49476) 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.165+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1981257386 learned_addr learned my addr 172.21.15.191:0/1981257386 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac59a0700 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329a000 0x5567331ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac59a0700 1 -- 172.21.15.191:0/102259287 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556732f81180 con 0x556732f06800 2024-04-27T04:35:49.183 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329a000 0x5567331ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1981257386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2e000 con 0x556732f07400 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/1981257386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x556733d990b0 tx=0x55673325cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf9d0c5ec116c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac319b700 1 -- 172.21.15.191:0/1981257386 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf600 con 0x556732f07400 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329a000 0x5567331ef080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556733d98f60 tx=0x55673325cc60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac319b700 1 -- 172.21.15.191:0/1981257386 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556732fa7c80 con 0x556732f07400 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2ac319b700 1 -- 172.21.15.191:0/1981257386 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf8c0 con 0x556732f07400 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1981257386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 msgr2=0x5567331eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1981257386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x556733d990b0 tx=0x55673325cae0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1981257386 shutdown_connections 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1981257386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1981257386 >> 172.21.15.191:0/1981257386 conn(0x556732f06c00 msgr2=0x556732ff3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1981257386 shutdown_connections 2024-04-27T04:35:49.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.166+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1981257386 wait complete. 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2aca9fca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2aca9fca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556732f428c0 con 0x556732f07400 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2ac91a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2ac91a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49486/0 (socket says 172.21.15.191:49486) 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 learned_addr learned my addr 172.21.15.191:0/885418539 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2eea0 con 0x556732f07400 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.167+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x556733d983f0 tx=0x55673325c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.168+0000 7f2ac1998700 1 -- 172.21.15.191:0/885418539 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf760 con 0x556732f07400 2024-04-27T04:35:49.185 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.168+0000 7f2ac1998700 1 -- 172.21.15.191:0/885418539 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556732fa7b00 con 0x556732f07400 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.168+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556732f2e820 con 0x556732f07400 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.168+0000 7f2ac1998700 1 -- 172.21.15.191:0/885418539 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556732fbf600 con 0x556732f07400 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.168+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556732f2e9c0 con 0x556732f07400 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.168+0000 7f2ac1998700 1 -- 172.21.15.191:0/885418539 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556733ca8dc0 con 0x556732f07400 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac1998700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329ac00 0x5567331ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac1998700 1 -- 172.21.15.191:0/885418539 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556732f81340 con 0x556732f07400 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329ac00 0x5567331ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556732f07800 0x5567331efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x556732f07800 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556732f07800 0x5567331efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.186 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329ac00 0x5567331ef600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5567332afd70 tx=0x5567333420c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556732f07800 0x5567331efb80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.169+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 realm 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b000 con 0x556732f07800 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b400 con 0x556732f07800 2024-04-27T04:35:49.187 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b800 con 0x556732f07800 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55673329bc00 0x5567331f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b000 con 0x55673329bc00 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.170+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55673329bc00 0x5567331f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55673329bc00 0x5567331f0100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.188 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2aca9fca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2aca9fca80 20 searching for the correct realm 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b400 con 0x55673329bc00 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55673329b400 0x5567331f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.171+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b000 con 0x55673329b400 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.172+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55673329b400 0x5567331f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.172+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55673329b400 0x5567331f0c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.172+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.172+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55673329b000 0x5567331f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.172+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.172+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55673329b000 0x5567331f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55673329b000 0x5567331f1180 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.173+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.174+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.174+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b400 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.174+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.174+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.190 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.174+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.174+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b400 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.175+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.191 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.176+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.192 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b400 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.177+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b400 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.178+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b400 2024-04-27T04:35:49.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329bc00 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.179+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b400 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:35:49.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x556732f07800 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x556732f07800 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358800 con 0x55673329bc00 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 zone default found 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 4 Realm: () 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 zonegroup default 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.180+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x55673329b000 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359000 con 0x55673329bc00 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359400 con 0x55673329bc00 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 started sync module instance, tier type = 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.181+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x55673329b400 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.182+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.182+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93901727619840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358800 con 0x55673329b400 2024-04-27T04:35:49.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.182+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359000 con 0x556732f07800 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.183+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93901727619840] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.184+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.184+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93901727620992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359400 con 0x556732f07800 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.184+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x55673329bc00 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.185+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93901727620992] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.185+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.185+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93901727622144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x55673329bc00 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.185+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x55673329b400 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.186+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.186+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93901727623296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55673329b800 con 0x55673329b400 2024-04-27T04:35:49.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.186+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x556732f07800 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.186+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93901727622144] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.187+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93901727623296] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.187+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.187+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93901727624448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x556732f07800 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.187+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358800 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.188+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93901727624448] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.188+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.189+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93901727625600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.189+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359000 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.189+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93901727625600] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.190+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.190+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93901727626752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359400 con 0x55673329b000 2024-04-27T04:35:49.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.190+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358000 con 0x55673329b000 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93901727626752] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93901727627904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x55673329b000 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=0 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=1 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=2 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=3 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=4 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=5 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.191+0000 7f2aca9fca80 20 add_watcher() i=6 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.192+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93901727627904] v33'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.192+0000 7f2aca9fca80 20 add_watcher() i=7 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.192+0000 7f2aca9fca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.192+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358800 con 0x556732f07800 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.192+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.192+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93901729185792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x556732f07800 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.193+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93901729185792] v33'229 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.193+0000 7f2aaf974700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.193+0000 7f2aca9fca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.193+0000 7f2aca9fca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aca9fca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aca9fca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556732fa83c0 con 0x556733359400 2024-04-27T04:35:49.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2ac81a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2ac81a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49490/0 (socket says 172.21.15.191:49490) 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2ac81a5700 1 -- 172.21.15.191:0/3517798753 learned_addr learned my addr 172.21.15.191:0/3517798753 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2ac81a5700 1 -- 172.21.15.191:0/3517798753 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2eb60 con 0x556733359400 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/3517798753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x5567332af440 tx=0x55673325df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5123444c80263d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aaf173700 1 -- 172.21.15.191:0/3517798753 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556733378000 con 0x556733359400 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aaf173700 1 -- 172.21.15.191:0/3517798753 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556733382000 con 0x556733359400 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.194+0000 7f2aaf173700 1 -- 172.21.15.191:0/3517798753 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556733378160 con 0x556733359400 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3517798753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 msgr2=0x5567331f0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/3517798753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x5567332af440 tx=0x55673325df80 comp rx=0 tx=0).stop 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3517798753 shutdown_connections 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/3517798753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3517798753 >> 172.21.15.191:0/3517798753 conn(0x556733359000 msgr2=0x556732ff4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3517798753 shutdown_connections 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 -- 172.21.15.191:0/3517798753 wait complete. 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.195+0000 7f2aca9fca80 1 Processor -- start 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aca9fca80 1 -- start start 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aca9fca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aca9fca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556732fa8500 con 0x556733359400 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2ac81a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2ac81a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:49498/0 (socket says 172.21.15.191:49498) 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 learned_addr learned my addr 172.21.15.191:0/1908283722 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556732f2ed00 con 0x556733359400 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55673337a210 tx=0x5567333424e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aad970700 1 -- 172.21.15.191:0/1908283722 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5567333782c0 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aad970700 1 -- 172.21.15.191:0/1908283722 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556733382180 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556732f2e340 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aad970700 1 -- 172.21.15.191:0/1908283722 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556733378000 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.196+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556732f2eb60 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2aad970700 1 -- 172.21.15.191:0/1908283722 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556733ca9b80 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2aad970700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5567333fa000 0x5567331f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2aad970700 1 -- 172.21.15.191:0/1908283722 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556732f81500 con 0x556733359400 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5567333fa000 0x5567331f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556733358000 0x556733410000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x556733358000 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.197+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556733358000 0x556733410000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.198+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5567333fa000 0x5567331f1700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5567332f3170 tx=0x556733342360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.198+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556733358000 0x556733410000 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.199+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'60 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.199+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556733358800 0x556733410580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.199+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fa400 con 0x556733358800 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.199+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556733358800 0x556733410580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.199+0000 7f2ac81a5700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556733358800 0x556733410580 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.200+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.200+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5567333fa800 0x556733410b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.200+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fac00 con 0x5567333fa800 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.200+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5567333fa800 0x556733410b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.200+0000 7f2ac91a7700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5567333fa800 0x556733410b00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.201+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.201+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb000 con 0x556733358000 2024-04-27T04:35:49.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.202+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'230 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.202+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5567333fb400 0x556733411080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.202+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb800 con 0x5567333fb400 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.202+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5567333fb400 0x556733411080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.202+0000 7f2ac89a6700 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5567333fb400 0x556733411080 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.203+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'80 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.203+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x5567333fa800 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.204+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'60 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.204+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb000 con 0x556733358000 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.205+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.205+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x556733358800 2024-04-27T04:35:49.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.207+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'55 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.207+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x5567333fb400 2024-04-27T04:35:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.207+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.208+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x5567333fa800 2024-04-27T04:35:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.208+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.208+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440000 con 0x5567333fb400 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.209+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'81 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.209+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440400 con 0x556733358800 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.210+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'33 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.210+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440800 con 0x5567333fb400 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.211+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'39 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.211+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440c00 con 0x556733358800 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.212+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'70 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.212+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441000 con 0x556733358000 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.213+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'231 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.213+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441400 con 0x5567333fa800 2024-04-27T04:35:49.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.214+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'71 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.214+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441800 con 0x5567333fb400 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.215+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'82 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.215+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441c00 con 0x5567333fa800 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.216+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'39 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.216+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440000 con 0x556733358000 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.217+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.217+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440800 con 0x556733358000 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.218+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'232 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.218+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440400 con 0x5567333fa800 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.219+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'72 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.219+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x5567333fb400 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.220+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'43 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.220+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.221+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.221+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb800 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.222+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'45 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.222+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb000 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.223+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.223+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fa400 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.223+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'61 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.223+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x556733358800 2024-04-27T04:35:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.225+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.225+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441400 con 0x5567333fa800 2024-04-27T04:35:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.225+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'39 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.225+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441c00 con 0x556733358800 2024-04-27T04:35:49.236 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.226+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'56 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.226+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440400 con 0x5567333fa800 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.227+0000 7f2ac91a7700 1 -- 172.21.15.191:0/1908283722 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'73 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733354b40 con 0x5567333fa800 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.227+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x556733358800 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.228+0000 7f2ac81a5700 1 -- 172.21.15.191:0/1908283722 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'71 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355440 con 0x556733358800 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.228+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x5567333fb400 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.229+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x556733355200 con 0x5567333fb400 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.230+0000 7f2aab16b700 20 reqs_thread_entry: start 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.230+0000 7f2aca9fca80 20 init_complete bucket index max shards: 11 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aa9167700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:19 2024 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aa8945700 20 reqs_thread_entry: start 2024-04-27T04:35:49.237 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x55673329b000 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.231+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x55673329b000 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(63 fud.buckets [call out=184b] v0'0 uv5 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 : miss 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.238 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.5 8:a1c59f76:root::.bucket.meta.myfoo7%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x556732f07800 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(64 .bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 20 rados_obj.operate() r=0 bl.length=353 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 info.flags=0x17 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 to cache LRU end 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.232+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 4.35 4:af759c8e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441000 con 0x556732f07800 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.233+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.233+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 4.35 4:af5ebd91:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441800 con 0x556732f07800 2024-04-27T04:35:49.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.233+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.233+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 4.1b 4:dab43c68:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440000 con 0x55673329b400 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.233+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2 [call out=85b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.233+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 4.1 4:83c51f26:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440800 con 0x55673329b400 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 4.3a 4:5f1aee09:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x55673329bc00 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:70 4.10 4:09efc34b:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x55673329bc00 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 [call out=377b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+377 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2aca9fca80 10 cls_bucket_list_unordered: got fourteen 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.234+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:71 4.28 4:1487299a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x556732f07800 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.235+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.235+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:72 4.8 4:110048bb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb800 con 0x55673329bc00 2024-04-27T04:35:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.235+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.235+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 4.6 4:6306db9f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb000 con 0x55673329bc00 2024-04-27T04:35:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.235+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.235+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:74 4.37 4:ecfa4e5d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fa400 con 0x55673329b000 2024-04-27T04:35:49.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.236+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.236+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.3d 4:be81c8c1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x55673329bc00 2024-04-27T04:35:49.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.236+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.236+0000 7f2aca9fca80 20 get_obj_state: rctx=0x556733331a40 obj=myfoo7:fourteen state=0x5567334c61e8 s->prefetch_data=0 2024-04-27T04:35:49.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.236+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 3.5s0 3:a3833d0d:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440c00 con 0x55673329b400 2024-04-27T04:35:49.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(76 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8_fourteen [getxattrs out=854b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+870 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2aca9fca80 10 manifest: total_size = 8 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2aca9fca80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.5241603051290075197 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2aca9fca80 20 get_obj_state: rctx=0x556733331a40 obj=myfoo7:fourteen state=0x5567334c61e8 s->prefetch_data=0 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2aca9fca80 20 prepare_atomic_modification: state is not atomic. state=0x5567334c61e8 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2aca9fca80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.237+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:77 4.10 4:09efc34b:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441c00 con 0x55673329bc00 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.238+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(77 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 [stat,call,call] v33'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.238+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 3.5s0 3:a3833d0d:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440000 con 0x55673329b400 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(78 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8_fourteen [call] v33'2 uv2 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:79 4.10 4:09efc34b:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440400 con 0x55673329bc00 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 20 list_objects_ordered: starting attempt 1 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T04:35:49.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:80 4.35 4:af759c8e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x556732f07800 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:81 4.35 4:af5ebd91:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441c00 con 0x556732f07800 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:82 4.1b 4:dab43c68:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440400 con 0x55673329b400 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:83 4.1 4:83c51f26:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb800 con 0x55673329b400 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:84 4.3a 4:5f1aee09:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb000 con 0x55673329bc00 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:85 4.10 4:09efc34b:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x55673329bc00 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:86 4.28 4:1487299a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x556732f07800 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:87 4.8 4:110048bb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x55673329bc00 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:88 4.6 4:6306db9f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544000 con 0x55673329bc00 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:89 4.37 4:ecfa4e5d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544400 con 0x55673329b000 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.239+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 4.3d 4:be81c8c1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544800 con 0x55673329bc00 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(84 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(82 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2 [call out=85b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(83 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(89 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(80 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(86 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(81 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(88 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(90 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.240+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(87 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(79 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 [stat,call,call] v33'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2ac299a700 20 handle_completion(): completion ok for obj=fourteen 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(85 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T04:35:49.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8])"; uploads.size()=0; is_truncated=0 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo7[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8]) 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:91 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544c00 con 0x55673329b400 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(91 myfoo7 [call out=48b,stat out=16b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv46 ondisk = 0) v8 ==== 276+0+305 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x17 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 lifecycle: adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.241+0000 7f2aca9fca80 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.242+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:92 8.5 8:a1c59f76:root::.bucket.meta.myfoo7%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545400 con 0x556732f07800 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(92 .bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 [delete,create,call,call,writefull 0~353,setxattr (123)] v33'6 uv6 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 info.flags=0x17 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 to cache LRU end 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 10 lifecycle: distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8, ofs0, ns] 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:93 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93901729189248 in=739b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545c00 con 0x55673329b000 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== watch-notify(notify (1) cookie 93901727626752 notify 141733920806 ret 0) v3 ==== 769+0+0 (crc 0 0 0) 0x5567333c5d40 con 0x55673329b000 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac299a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920806 cookie 93901727626752 notifier 4828 bl.length()=727 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(93 notify.6 [notify cookie 93901729189248 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac299a700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 info.flags=0x17 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac299a700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 to cache LRU end 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac299a700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac299a700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:94 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441400 con 0x55673329b000 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(94 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== watch-notify(notify_complete (2) cookie 93901729189248 notify 141733920806 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5567332f7a00 con 0x55673329b000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 20 lifecycle: handle_bi_update(): bucket=:myfoo7[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8]): orig_sources= new_sources= 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 20 lifecycle: handle_bi_update(): bucket=:myfoo7[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8]): potential sources added=[] removed=[] 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 20 lifecycle: handle_bi_update(): bucket=:myfoo7[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8]): orig_dests= new_dests= 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 20 lifecycle: handle_bi_update(): bucket=:myfoo7[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8]): potential dests added=[] removed=[] 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.243+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:33 6.17 6:ee62d5d2:lc::lc.1:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544c00 con 0x556733358000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.244+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(33 lc.1 [call] v33'13 uv13 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.245+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:34 6.17 6:ee62d5d2:lc::lc.1:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544400 con 0x556733358000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.245+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(34 lc.1 [call] v33'14 uv14 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.245+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:35 6.17 6:ee62d5d2:lc::lc.1:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x556733358000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(35 lc.1 [call] v33'15 uv15 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x556733355200 con 0x556733358000 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:95 4.35 4:af759c8e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x556732f07800 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:96 4.35 4:af5ebd91:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x556732f07800 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:97 4.1b 4:dab43c68:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440800 con 0x55673329b400 2024-04-27T04:35:49.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:98 4.1 4:83c51f26:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440c00 con 0x55673329b400 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:99 4.3a 4:5f1aee09:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440000 con 0x55673329bc00 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:100 4.10 4:09efc34b:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440400 con 0x55673329bc00 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:101 4.28 4:1487299a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441000 con 0x556732f07800 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:102 4.8 4:110048bb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441800 con 0x55673329bc00 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:103 4.6 4:6306db9f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441c00 con 0x55673329bc00 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:104 4.37 4:ecfa4e5d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x55673329b000 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.246+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:105 4.3d 4:be81c8c1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb800 con 0x55673329bc00 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(97 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2 [call out=85b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(98 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(101 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(104 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(99 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.257 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 29 ==== osd_op_reply(103 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 30 ==== osd_op_reply(100 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 31 ==== osd_op_reply(105 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 32 ==== osd_op_reply(102 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733355200 con 0x55673329bc00 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(95 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(96 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.247+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:106 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fa400 con 0x55673329b000 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.248+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== osd_op_reply(106 fud.buckets [call] v33'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.248+0000 7f2aca9fca80 10 cache get: name=default.rgw.meta+root+myfoo7 : hit (requested=0x11, cached=0x17) 2024-04-27T04:35:49.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.248+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:107 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x55673329b400 2024-04-27T04:35:49.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.249+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== osd_op_reply(107 myfoo7 [call,call,delete] v33'49 uv49 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.249+0000 7f2aca9fca80 10 removing default.rgw.meta+root+myfoo7 from cache 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.249+0000 7f2aca9fca80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.249+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 93901729189248 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x55673329b400 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== watch-notify(notify (1) cookie 93901727623296 notify 141733920800 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x5567332f7ba0 con 0x55673329b400 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2ac319b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920800 cookie 93901727623296 notifier 4828 bl.length()=150 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2ac319b700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544000 con 0x55673329b400 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(108 notify.3 [notify cookie 93901729189248 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== watch-notify(notify_complete (2) cookie 93901729189248 notify 141733920800 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5567332f61a0 con 0x55673329b400 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.250+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:110 8.5 8:a1c59f76:root::.bucket.meta.myfoo7%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x556732f07800 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(110 .bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 [call,call,delete] v33'7 uv7 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2aca9fca80 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 from cache 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2aca9fca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8, ofs0, ns] 2024-04-27T04:35:49.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93901729189248 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359800 con 0x55673329b000 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== watch-notify(notify (1) cookie 93901727626752 notify 141733920807 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x5567332f7a00 con 0x55673329b000 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2ac299a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920807 cookie 93901727626752 notifier 4828 bl.length()=207 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2ac299a700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544800 con 0x55673329b000 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== osd_op_reply(111 notify.6 [notify cookie 93901729189248 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.251+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(112 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== watch-notify(notify_complete (2) cookie 93901729189248 notify 141733920807 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5567333c44e0 con 0x55673329b000 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:113 4.35 4:af759c8e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fa400 con 0x556732f07800 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:114 4.35 4:af5ebd91:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb000 con 0x556732f07800 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:115 4.1b 4:dab43c68:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x55673329b400 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:116 4.1 4:83c51f26:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x55673329b400 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:117 4.3a 4:5f1aee09:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441400 con 0x55673329bc00 2024-04-27T04:35:49.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:118 4.10 4:09efc34b:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441000 con 0x55673329bc00 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:119 4.28 4:1487299a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544c00 con 0x556732f07800 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:120 4.8 4:110048bb:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545000 con 0x55673329bc00 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:121 4.6 4:6306db9f:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545400 con 0x55673329bc00 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:122 4.37 4:ecfa4e5d:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545800 con 0x55673329b000 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.252+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:123 4.3d 4:be81c8c1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545c00 con 0x55673329bc00 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.253+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(115 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.2 [delete] v33'7 uv7 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.253+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(113 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.0 [delete] v33'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.253+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 27 ==== osd_op_reply(116 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.3 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 34 ==== osd_op_reply(114 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.1 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 35 ==== osd_op_reply(119 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.6 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 33 ==== osd_op_reply(117 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.4 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 34 ==== osd_op_reply(118 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.5 [delete] v33'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== osd_op_reply(122 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.9 [delete] v33'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 35 ==== osd_op_reply(121 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.8 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 36 ==== osd_op_reply(123 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.10 [delete] v33'4 uv4 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 37 ==== osd_op_reply(120 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8.7 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 : miss 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 20 rados->read ofs=0 len=0 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:124 6.12 6:4f7483ff:::pubsub..bucket.myfoo7%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556732f07c00 con 0x55673329b400 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 28 ==== osd_op_reply(124 pubsub..bucket.myfoo7/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 info.flags=0x0 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 10 adding default.rgw.log++pubsub..bucket.myfoo7/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 to cache LRU end 2024-04-27T04:35:49.271 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.254+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:125 6.12 6:4f7483ff:::pubsub..bucket.myfoo7%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fb800 con 0x55673329b400 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.255+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 29 ==== osd_op_reply(125 pubsub..bucket.myfoo7/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.8 [delete] v33'28 uv21 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.255+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:126 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x55673329b000 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.256+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== osd_op_reply(126 fud.buckets [call] v33'7 uv7 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.256+0000 7f2aca9fca80 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.256+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:127 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545000 con 0x55673329b000 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(127 p5YnriCv1nAtykxBrupQ [delete] v33'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2aca9fca80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:128 7.15 7:a93a5511:::notify.2:head [notify cookie 93901729189248 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545400 con 0x55673329bc00 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 38 ==== watch-notify(notify (1) cookie 93901727622144 notify 141733920777 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x5567333c3ba0 con 0x55673329bc00 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2ac319b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920777 cookie 93901727622144 notifier 4828 bl.length()=170 2024-04-27T04:35:49.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 39 ==== osd_op_reply(128 notify.2 [notify cookie 93901729189248 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.257+0000 7f2ac319b700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:129 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544400 con 0x55673329bc00 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.258+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 40 ==== osd_op_reply(129 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.258+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 41 ==== watch-notify(notify_complete (2) cookie 93901729189248 notify 141733920777 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5567333bf1e0 con 0x55673329bc00 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.258+0000 7f2aca9fca80 10 removing email index: 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.258+0000 7f2aca9fca80 10 removing user buckets index 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.258+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:130 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544000 con 0x55673329b000 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 33 ==== osd_op_reply(130 fud.buckets [delete] v33'8 uv8 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2aca9fca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:131 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93901729189248 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358c00 con 0x55673329b000 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 34 ==== watch-notify(notify (1) cookie 93901727626752 notify 141733920808 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x5567333c44e0 con 0x55673329b000 2024-04-27T04:35:49.273 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac299a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920808 cookie 93901727626752 notifier 4828 bl.length()=160 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac299a700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:132 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fa400 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 35 ==== osd_op_reply(131 notify.6 [notify cookie 93901729189248 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 36 ==== osd_op_reply(132 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 37 ==== watch-notify(notify_complete (2) cookie 93901729189248 notify 141733920808 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5567333c4680 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2aca9fca80 10 removing user index: fud 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.259+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:133 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733359c00 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 38 ==== osd_op_reply(133 fud [call,call,delete] v33'7 uv7 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2aca9fca80 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2aca9fca80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 93901729189248 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733358400 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 39 ==== watch-notify(notify (1) cookie 93901727625600 notify 141733920809 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x5567333c4680 con 0x55673329b000 2024-04-27T04:35:49.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2ac319b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920809 cookie 93901727625600 notifier 4828 bl.length()=152 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2ac319b700 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440000 con 0x55673329b000 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.260+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 40 ==== osd_op_reply(134 notify.5 [notify cookie 93901729189248 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.261+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 41 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.261+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 42 ==== watch-notify(notify_complete (2) cookie 93901729189248 notify 141733920809 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5567333c4820 con 0x55673329b000 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.261+0000 7f2aaf974700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.261+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:136 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93901729185792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545800 con 0x556732f07800 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.262+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 36 ==== osd_op_reply(136 data_loggenerations_metadata [watch unwatch cookie 93901729185792] v33'233 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.262+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:137 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93901727619840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733545000 con 0x55673329b400 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.263+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 30 ==== osd_op_reply(137 notify.0 [watch unwatch cookie 93901727619840] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.263+0000 7f2aca9fca80 20 remove_watcher() i=0 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.263+0000 7f2aca9fca80 2 removed watcher, disabling cache 2024-04-27T04:35:49.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.263+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:138 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93901727620992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733544000 con 0x556732f07800 2024-04-27T04:35:49.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.265+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 37 ==== osd_op_reply(138 notify.1 [watch unwatch cookie 93901727620992] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.265+0000 7f2aca9fca80 20 remove_watcher() i=1 2024-04-27T04:35:49.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.265+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:139 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93901727622144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5567333fbc00 con 0x55673329bc00 2024-04-27T04:35:49.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.266+0000 7f2ac89a6700 1 -- 172.21.15.191:0/885418539 <== osd.3 v2:172.21.15.191:6800/41887 42 ==== osd_op_reply(139 notify.2 [watch unwatch cookie 93901727622144] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355680 con 0x55673329bc00 2024-04-27T04:35:49.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.266+0000 7f2aca9fca80 20 remove_watcher() i=2 2024-04-27T04:35:49.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.266+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:140 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93901727623296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733440c00 con 0x55673329b400 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.267+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 <== osd.2 v2:172.21.15.191:6816/41888 31 ==== osd_op_reply(140 notify.3 [watch unwatch cookie 93901727623296] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733355440 con 0x55673329b400 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.267+0000 7f2aca9fca80 20 remove_watcher() i=3 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.267+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93901727624448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441000 con 0x556732f07800 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.268+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.1 v2:172.21.15.191:6808/41886 38 ==== osd_op_reply(141 notify.4 [watch unwatch cookie 93901727624448] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x556732f07800 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.268+0000 7f2aca9fca80 20 remove_watcher() i=4 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.268+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93901727625600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441400 con 0x55673329b000 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.269+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 43 ==== osd_op_reply(142 notify.5 [watch unwatch cookie 93901727625600] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.269+0000 7f2aca9fca80 20 remove_watcher() i=5 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.269+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:143 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93901727626752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441800 con 0x55673329b000 2024-04-27T04:35:49.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.270+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 44 ==== osd_op_reply(143 notify.6 [watch unwatch cookie 93901727626752] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.270+0000 7f2aca9fca80 20 remove_watcher() i=6 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.270+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:144 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93901727627904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556733441c00 con 0x55673329b000 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2ac91a7700 1 -- 172.21.15.191:0/885418539 <== osd.0 v2:172.21.15.138:6800/39496 45 ==== osd_op_reply(144 notify.7 [watch unwatch cookie 93901727627904] v33'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556733354b40 con 0x55673329b000 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 20 remove_watcher() i=7 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55673329b000 msgr2=0x5567331f1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55673329b000 0x5567331f1180 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556732f07800 msgr2=0x5567331efb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556732f07800 0x5567331efb80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55673329b400 msgr2=0x5567331f0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55673329b400 0x5567331f0c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55673329bc00 msgr2=0x5567331f0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55673329bc00 0x5567331f0100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329ac00 msgr2=0x5567331ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329ac00 0x5567331ef600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5567332afd70 tx=0x5567333420c0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 msgr2=0x5567331eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/885418539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f07400 0x5567331eeb00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x556733d983f0 tx=0x55673325c5a0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 reap_dead start 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2ac81a5700 1 -- 172.21.15.191:0/885418539 reap_dead start 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 shutdown_connections 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 >> 172.21.15.191:0/885418539 conn(0x556732f06c00 msgr2=0x556732ff3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 shutdown_connections 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.271+0000 7f2aca9fca80 1 -- 172.21.15.191:0/885418539 wait complete. 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556733358800 msgr2=0x556733410580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x556733358800 0x556733410580 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556733358000 msgr2=0x556733410000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556733358000 0x556733410000 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5567333fb400 msgr2=0x556733411080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x5567333fb400 0x556733411080 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5567333fa800 msgr2=0x556733410b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x5567333fa800 0x556733410b00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5567333fa000 msgr2=0x5567331f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x5567333fa000 0x5567331f1700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5567332f3170 tx=0x556733342360 comp rx=0 tx=0).stop 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 msgr2=0x5567331f0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2ac89a6700 1 -- 172.21.15.191:0/1908283722 reap_dead start 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55673337a210 tx=0x5567333424e0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 shutdown_connections 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/1908283722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733359400 0x5567331f0680 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.284 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 >> 172.21.15.191:0/1908283722 conn(0x556733359000 msgr2=0x556732ff4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 shutdown_connections 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/1908283722 wait complete. 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329a000 msgr2=0x5567331ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329a000 0x5567331ef080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556733d98f60 tx=0x55673325cc60 comp rx=0 tx=0).stop 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 msgr2=0x5567331ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x556733cdf980 tx=0x556732ff7920 comp rx=0 tx=0).stop 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 shutdown_connections 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55673329a000 0x5567331ef080 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 --2- 172.21.15.191:0/102259287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556732f06800 0x5567331ee580 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:35:49.285 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 >> 172.21.15.191:0/102259287 conn(0x556732f06400 msgr2=0x556732ff3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:35:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 shutdown_connections 2024-04-27T04:35:49.286 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:35:49.272+0000 7f2aca9fca80 1 -- 172.21.15.191:0/102259287 wait complete. 2024-04-27T04:36:29.288 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-04-27T04:36:29.289 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-04-27T04:36:29.289 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-04-27T04:36:29.342 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:29.342 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:29.351 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.341+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.341+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.341+0000 7fa80c709a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.341+0000 7fa80c709a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555bd5283500 con 0x555bd4f58c00 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.341+0000 7fa80c709a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x555bd4ffaa00 con 0x555bd4f58800 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa80a6b3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa809eb2700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x555bd4f58c00 0x555bd5044800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46330/0 (socket says 172.21.15.191:46330) 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa809eb2700 1 -- 172.21.15.191:0/469181881 learned_addr learned my addr 172.21.15.191:0/469181881 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa80a6b3700 1 -- 172.21.15.191:0/469181881 >> v1:172.21.15.138:6789/0 conn(0x555bd4f58c00 legacy=0x555bd5044800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa80a6b3700 1 -- 172.21.15.191:0/469181881 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f81d40 con 0x555bd4f58800 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa8096b1700 1 -- 172.21.15.191:0/469181881 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792362520 0 0) 0x555bd5d26000 con 0x555bd4f58c00 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/469181881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x555bd523b9e0 tx=0x555bd5049920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f9e1c130c1aaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa8096b1700 1 -- 172.21.15.191:0/469181881 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd50114a0 con 0x555bd4f58800 2024-04-27T04:36:29.352 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa8096b1700 1 -- 172.21.15.191:0/469181881 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd5283800 con 0x555bd4f58800 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.342+0000 7fa8096b1700 1 -- 172.21.15.191:0/469181881 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd5011600 con 0x555bd4f58800 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 -- 172.21.15.191:0/469181881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 msgr2=0x555bd5240580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 --2- 172.21.15.191:0/469181881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x555bd523b9e0 tx=0x555bd5049920 comp rx=0 tx=0).stop 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 -- 172.21.15.191:0/469181881 shutdown_connections 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 --2- 172.21.15.191:0/469181881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 -- 172.21.15.191:0/469181881 >> 172.21.15.191:0/469181881 conn(0x555bd4f58400 msgr2=0x555bd5044000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 -- 172.21.15.191:0/469181881 shutdown_connections 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.343+0000 7fa80c709a80 1 -- 172.21.15.191:0/469181881 wait complete. 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80c709a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80c709a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd4ffa500 con 0x555bd4f58800 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80a6b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.353 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80a6b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40934/0 (socket says 172.21.15.191:40934) 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80a6b3700 1 -- 172.21.15.191:0/1728774848 learned_addr learned my addr 172.21.15.191:0/1728774848 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80a6b3700 1 -- 172.21.15.191:0/1728774848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f81ba0 con 0x555bd4f58800 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.344+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/1728774848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x555bd5d29800 tx=0x555bd5049800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db9c8878952a3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa808eb0700 1 -- 172.21.15.191:0/1728774848 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd5011760 con 0x555bd4f58800 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa808eb0700 1 -- 172.21.15.191:0/1728774848 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd5283b00 con 0x555bd4f58800 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa808eb0700 1 -- 172.21.15.191:0/1728774848 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd50111e0 con 0x555bd4f58800 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 -- 172.21.15.191:0/1728774848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 msgr2=0x555bd5240580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 --2- 172.21.15.191:0/1728774848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x555bd5d29800 tx=0x555bd5049800 comp rx=0 tx=0).stop 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 -- 172.21.15.191:0/1728774848 shutdown_connections 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 --2- 172.21.15.191:0/1728774848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 -- 172.21.15.191:0/1728774848 >> 172.21.15.191:0/1728774848 conn(0x555bd4f58400 msgr2=0x555bd5045000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 -- 172.21.15.191:0/1728774848 shutdown_connections 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.345+0000 7fa80c709a80 1 -- 172.21.15.191:0/1728774848 wait complete. 2024-04-27T04:36:29.354 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80c709a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80c709a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd4ffa3c0 con 0x555bd4f58800 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80a6b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80a6b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40938/0 (socket says 172.21.15.191:40938) 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80a6b3700 1 -- 172.21.15.191:0/3680896602 learned_addr learned my addr 172.21.15.191:0/3680896602 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.355 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.346+0000 7fa80a6b3700 1 -- 172.21.15.191:0/3680896602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f81a00 con 0x555bd4f58800 2024-04-27T04:36:29.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x555bd5d29ec0 tx=0x555bd5049920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa8076ad700 1 -- 172.21.15.191:0/3680896602 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd50114a0 con 0x555bd4f58800 2024-04-27T04:36:29.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa8076ad700 1 -- 172.21.15.191:0/3680896602 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd5283c80 con 0x555bd4f58800 2024-04-27T04:36:29.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555bd4f81860 con 0x555bd4f58800 2024-04-27T04:36:29.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa8076ad700 1 -- 172.21.15.191:0/3680896602 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd5011600 con 0x555bd4f58800 2024-04-27T04:36:29.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bd4f816c0 con 0x555bd4f58800 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.347+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa8076ad700 1 -- 172.21.15.191:0/3680896602 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555bd5cfab00 con 0x555bd4f58800 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80c709a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa8076ad700 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0000 0x555bd5241080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80c709a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd4f94640 con 0x555bd4f59400 2024-04-27T04:36:29.357 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa8076ad700 1 -- 172.21.15.191:0/3680896602 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x555bd4fd3180 con 0x555bd4f58800 2024-04-27T04:36:29.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0000 0x555bd5241080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80aeb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80aeb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40952/0 (socket says 172.21.15.191:40952) 2024-04-27T04:36:29.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80aeb4700 1 -- 172.21.15.191:0/3551931556 learned_addr learned my addr 172.21.15.191:0/3551931556 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0000 0x555bd5241080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x555bd5dd1050 tx=0x555bd52a6b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80aeb4700 1 -- 172.21.15.191:0/3551931556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f80000 con 0x555bd4f59400 2024-04-27T04:36:29.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.348+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/3551931556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x555bd5dd03f0 tx=0x555bd52a6e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7ad9f1cf0c40fb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa804ea8700 1 -- 172.21.15.191:0/3551931556 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd50114a0 con 0x555bd4f59400 2024-04-27T04:36:29.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa804ea8700 1 -- 172.21.15.191:0/3551931556 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd4ff9c80 con 0x555bd4f59400 2024-04-27T04:36:29.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa804ea8700 1 -- 172.21.15.191:0/3551931556 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd5011600 con 0x555bd4f59400 2024-04-27T04:36:29.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 -- 172.21.15.191:0/3551931556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 msgr2=0x555bd5240b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 --2- 172.21.15.191:0/3551931556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x555bd5dd03f0 tx=0x555bd52a6e40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 -- 172.21.15.191:0/3551931556 shutdown_connections 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 --2- 172.21.15.191:0/3551931556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 -- 172.21.15.191:0/3551931556 >> 172.21.15.191:0/3551931556 conn(0x555bd4f58c00 msgr2=0x555bd5045800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 -- 172.21.15.191:0/3551931556 shutdown_connections 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 -- 172.21.15.191:0/3551931556 wait complete. 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.349+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.360 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80c709a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80c709a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd4f948c0 con 0x555bd4f59400 2024-04-27T04:36:29.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80aeb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80aeb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40964/0 (socket says 172.21.15.191:40964) 2024-04-27T04:36:29.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 learned_addr learned my addr 172.21.15.191:0/590289702 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.361 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f80ea0 con 0x555bd4f59400 2024-04-27T04:36:29.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.350+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x555bd5dd0210 tx=0x555bd52a6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.351+0000 7fa8036a5700 1 -- 172.21.15.191:0/590289702 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd5011760 con 0x555bd4f59400 2024-04-27T04:36:29.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.351+0000 7fa8036a5700 1 -- 172.21.15.191:0/590289702 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd4ff9b00 con 0x555bd4f59400 2024-04-27T04:36:29.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.351+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555bd4f80820 con 0x555bd4f59400 2024-04-27T04:36:29.399 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.351+0000 7fa8036a5700 1 -- 172.21.15.191:0/590289702 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd50114a0 con 0x555bd4f59400 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.351+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bd4f809c0 con 0x555bd4f59400 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.351+0000 7fa8036a5700 1 -- 172.21.15.191:0/590289702 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555bd5cfadc0 con 0x555bd4f59400 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa8036a5700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0c00 0x555bd5241600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa8036a5700 1 -- 172.21.15.191:0/590289702 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x555bd4fd3340 con 0x555bd4f59400 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa809eb2700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0c00 0x555bd5241600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa809eb2700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0c00 0x555bd5241600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x555bd5d28030 tx=0x555bd53900c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555bd4f59800 0x555bd5241b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd4f59c00 con 0x555bd4f59800 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.352+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555bd4f59800 0x555bd5241b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555bd4f59800 0x555bd5241b80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 realm 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1000 con 0x555bd4f59800 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:36:29.401 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1400 con 0x555bd4f59800 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.353+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1800 con 0x555bd4f59800 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.402 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555bd52e1c00 0x555bd5242100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1000 con 0x555bd52e1c00 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555bd52e1c00 0x555bd5242100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555bd52e1c00 0x555bd5242100 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 20 searching for the correct realm 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.354+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1400 con 0x555bd52e1c00 2024-04-27T04:36:29.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.355+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.355+0000 7fa80a6b3700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555bd52e1400 0x555bd5242c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.355+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1000 con 0x555bd52e1400 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.355+0000 7fa809eb2700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555bd52e1400 0x555bd5242c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.355+0000 7fa809eb2700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555bd52e1400 0x555bd5242c00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.356+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.356+0000 7fa809eb2700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555bd52e1000 0x555bd5243180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.356+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.356+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555bd52e1000 0x555bd5243180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.356+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555bd52e1000 0x555bd5243180 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.357+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1400 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.358+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1400 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.359+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.360+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.361+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1400 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.362+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1400 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1400 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1c00 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.363+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1400 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:36:29.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd4f59c00 con 0x555bd4f59800 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa400 con 0x555bd4f59800 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.364+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa800 con 0x555bd52e1c00 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:36:29.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 zone default found 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 4 Realm: () 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 zonegroup default 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aac00 con 0x555bd52e1000 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab000 con 0x555bd52e1c00 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.365+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab400 con 0x555bd52e1c00 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.366+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.366+0000 7fa80c709a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.366+0000 7fa80c709a80 20 started sync module instance, tier type = 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.366+0000 7fa80c709a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.366+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab800 con 0x555bd52e1400 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.367+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.367+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93852906257152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa800 con 0x555bd52e1400 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.367+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab000 con 0x555bd4f59800 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.368+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93852906257152] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.368+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.368+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93852906258304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab400 con 0x555bd4f59800 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.368+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa400 con 0x555bd52e1c00 2024-04-27T04:36:29.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.369+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.370+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93852906259456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aac00 con 0x555bd52e1c00 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.370+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd4f59c00 con 0x555bd52e1400 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.370+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93852906258304] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.371+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.371+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93852906259456] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.371+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93852906260608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd52e1800 con 0x555bd52e1400 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.371+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd4f59800 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.372+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.372+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93852906260608] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.372+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93852906261760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab800 con 0x555bd4f59800 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.372+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa800 con 0x555bd52e1000 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.373+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93852906261760] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.373+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.373+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93852906262912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53abc00 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.373+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab000 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.374+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93852906262912] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.374+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.374+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93852906264064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab400 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.374+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa000 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93852906264064] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93852906265216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab800 con 0x555bd52e1000 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=0 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=1 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=2 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=3 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=4 2024-04-27T04:36:29.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=5 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.375+0000 7fa80c709a80 20 add_watcher() i=6 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.376+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93852906265216] v33'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.376+0000 7fa80c709a80 20 add_watcher() i=7 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.376+0000 7fa80c709a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.376+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa800 con 0x555bd4f59800 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.377+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.377+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93852907823104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53abc00 con 0x555bd4f59800 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93852907823104] v33'234 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa7f1681700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80c709a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80c709a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80c709a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.378+0000 7fa80c709a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd4ffa3c0 con 0x555bd53ab400 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.379+0000 7fa809eb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.379+0000 7fa809eb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40976/0 (socket says 172.21.15.191:40976) 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.379+0000 7fa809eb2700 1 -- 172.21.15.191:0/4202431177 learned_addr learned my addr 172.21.15.191:0/4202431177 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.379+0000 7fa809eb2700 1 -- 172.21.15.191:0/4202431177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f80b60 con 0x555bd53ab400 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.379+0000 7fa809eb2700 1 --2- 172.21.15.191:0/4202431177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x555bd52e36b0 tx=0x555bd52a7f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ba646e2146fe49 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.379+0000 7fa7f0e80700 1 -- 172.21.15.191:0/4202431177 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd53ca000 con 0x555bd53ab400 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa7f0e80700 1 -- 172.21.15.191:0/4202431177 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd53d2000 con 0x555bd53ab400 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa7f0e80700 1 -- 172.21.15.191:0/4202431177 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd53ca160 con 0x555bd53ab400 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 -- 172.21.15.191:0/4202431177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 msgr2=0x555bd5242680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 --2- 172.21.15.191:0/4202431177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x555bd52e36b0 tx=0x555bd52a7f80 comp rx=0 tx=0).stop 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 -- 172.21.15.191:0/4202431177 shutdown_connections 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 --2- 172.21.15.191:0/4202431177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 -- 172.21.15.191:0/4202431177 >> 172.21.15.191:0/4202431177 conn(0x555bd53ab000 msgr2=0x555bd5046000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 -- 172.21.15.191:0/4202431177 shutdown_connections 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.380+0000 7fa80c709a80 1 -- 172.21.15.191:0/4202431177 wait complete. 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa80c709a80 1 Processor -- start 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa80c709a80 1 -- start start 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa80c709a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa80c709a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd4ffa500 con 0x555bd53ab400 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa809eb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa809eb2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40986/0 (socket says 172.21.15.191:40986) 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.381+0000 7fa809eb2700 1 -- 172.21.15.191:0/130534659 learned_addr learned my addr 172.21.15.191:0/130534659 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa809eb2700 1 -- 172.21.15.191:0/130534659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd4f80d00 con 0x555bd53ab400 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa809eb2700 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x555bd53bb200 tx=0x555bd53904e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa7ef67d700 1 -- 172.21.15.191:0/130534659 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd53ca2c0 con 0x555bd53ab400 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa7ef67d700 1 -- 172.21.15.191:0/130534659 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bd53d2180 con 0x555bd53ab400 2024-04-27T04:36:29.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555bd4f80340 con 0x555bd53ab400 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa7ef67d700 1 -- 172.21.15.191:0/130534659 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555bd53ca000 con 0x555bd53ab400 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.382+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bd4f80b60 con 0x555bd53ab400 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.383+0000 7fa7ef67d700 1 -- 172.21.15.191:0/130534659 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555bd5cfbb80 con 0x555bd53ab400 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.383+0000 7fa7ef67d700 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd544c000 0x555bd5243700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.383+0000 7fa7ef67d700 1 -- 172.21.15.191:0/130534659 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x555bd4fd3500 con 0x555bd53ab400 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.383+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd544c000 0x555bd5243700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.383+0000 7fa80c709a80 5 note: GC not initialized 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.383+0000 7fa80aeb4700 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd544c000 0x555bd5243700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x555bd52e3ad0 tx=0x555bd5390360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa7ece78700 20 reqs_thread_entry: start 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa80c709a80 20 init_complete bucket index max shards: 11 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa80c709a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa7eae74700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:59 2024 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa7ea652700 20 reqs_thread_entry: start 2024-04-27T04:36:29.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa80c709a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa80c709a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.384+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53ab800 con 0x555bd4f59800 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.385+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.385+0000 7fa80c709a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.385+0000 7fa80c709a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.385+0000 7fa80c709a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.385+0000 7fa80c709a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.385+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd53aa800 con 0x555bd52e1000 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.387+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call] v33'34 uv34 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.387+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544c400 con 0x555bd52e1000 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.388+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 usage.10 [call] v33'35 uv34 ondisk = -61 ((61) No data available)) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.388+0000 7fa7f1681700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.388+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93852907823104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544c800 con 0x555bd4f59800 2024-04-27T04:36:29.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.389+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93852907823104] v33'235 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.389+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93852906257152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544cc00 con 0x555bd52e1400 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.390+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93852906257152] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.390+0000 7fa80c709a80 20 remove_watcher() i=0 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.390+0000 7fa80c709a80 2 removed watcher, disabling cache 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.390+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93852906258304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544d000 con 0x555bd4f59800 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.392+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93852906258304] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.392+0000 7fa80c709a80 20 remove_watcher() i=1 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.392+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93852906259456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544d400 con 0x555bd52e1c00 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.393+0000 7fa80a6b3700 1 -- 172.21.15.191:0/590289702 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93852906259456] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3200 con 0x555bd52e1c00 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.393+0000 7fa80c709a80 20 remove_watcher() i=2 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.393+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93852906260608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544d800 con 0x555bd52e1400 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.394+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93852906260608] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a3440 con 0x555bd52e1400 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.394+0000 7fa80c709a80 20 remove_watcher() i=3 2024-04-27T04:36:29.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.394+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93852906261760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544dc00 con 0x555bd4f59800 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.395+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93852906261760] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd4f59800 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.395+0000 7fa80c709a80 20 remove_watcher() i=4 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.395+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93852906262912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544c400 con 0x555bd52e1000 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.396+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93852906262912] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.396+0000 7fa80c709a80 20 remove_watcher() i=5 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.396+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93852906264064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544c800 con 0x555bd52e1000 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.397+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93852906264064] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.397+0000 7fa80c709a80 20 remove_watcher() i=6 2024-04-27T04:36:29.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.397+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93852906265216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555bd544d000 con 0x555bd52e1000 2024-04-27T04:36:29.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.398+0000 7fa80aeb4700 1 -- 172.21.15.191:0/590289702 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93852906265216] v33'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555bd53a2b40 con 0x555bd52e1000 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.398+0000 7fa80c709a80 20 remove_watcher() i=7 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555bd52e1000 msgr2=0x555bd5243180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x555bd52e1000 0x555bd5243180 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555bd4f59800 msgr2=0x555bd5241b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x555bd4f59800 0x555bd5241b80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555bd52e1400 msgr2=0x555bd5242c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x555bd52e1400 0x555bd5242c00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555bd52e1c00 msgr2=0x555bd5242100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x555bd52e1c00 0x555bd5242100 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0c00 msgr2=0x555bd5241600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0c00 0x555bd5241600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x555bd5d28030 tx=0x555bd53900c0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 msgr2=0x555bd5240b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/590289702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f59400 0x555bd5240b00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x555bd5dd0210 tx=0x555bd52a6a20 comp rx=0 tx=0).stop 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 reap_dead start 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa809eb2700 1 -- 172.21.15.191:0/590289702 reap_dead start 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 shutdown_connections 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 >> 172.21.15.191:0/590289702 conn(0x555bd4f58c00 msgr2=0x555bd5045800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 shutdown_connections 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/590289702 wait complete. 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd544c000 msgr2=0x555bd5243700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd544c000 0x555bd5243700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x555bd52e3ad0 tx=0x555bd5390360 comp rx=0 tx=0).stop 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 msgr2=0x555bd5242680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x555bd53bb200 tx=0x555bd53904e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 shutdown_connections 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd544c000 0x555bd5243700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 --2- 172.21.15.191:0/130534659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd53ab400 0x555bd5242680 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 >> 172.21.15.191:0/130534659 conn(0x555bd53ab000 msgr2=0x555bd5046000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 shutdown_connections 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.399+0000 7fa80c709a80 1 -- 172.21.15.191:0/130534659 wait complete. 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0000 msgr2=0x555bd5241080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0000 0x555bd5241080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x555bd5dd1050 tx=0x555bd52a6b40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 msgr2=0x555bd5240580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x555bd5d29ec0 tx=0x555bd5049920 comp rx=0 tx=0).stop 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 shutdown_connections 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x555bd52e0000 0x555bd5241080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 --2- 172.21.15.191:0/3680896602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bd4f58800 0x555bd5240580 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 >> 172.21.15.191:0/3680896602 conn(0x555bd4f58400 msgr2=0x555bd5045000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 shutdown_connections 2024-04-27T04:36:29.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.400+0000 7fa80c709a80 1 -- 172.21.15.191:0/3680896602 wait complete. 2024-04-27T04:36:29.427 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-27T04:36:29.427 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-27T04:36:29.427 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-27T04:36:29.473 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:29.473 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.472+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.472+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.472+0000 7fbe6677ea80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.472+0000 7fbe6677ea80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bbaa3f7500 con 0x55bbaa0ccc00 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.472+0000 7fbe6677ea80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bbaa16ea00 con 0x55bbaa0cc800 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe64728700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63f27700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55bbaa0ccc00 0x55bbaa1b8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46340/0 (socket says 172.21.15.191:46340) 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63f27700 1 -- 172.21.15.191:0/195009927 learned_addr learned my addr 172.21.15.191:0/195009927 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63726700 1 -- 172.21.15.191:0/195009927 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167947859 0 0) 0x55bbaae9a000 con 0x55bbaa0ccc00 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63726700 1 -- 172.21.15.191:0/195009927 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55bbaa3f7680 con 0x55bbaa0ccc00 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe64728700 1 -- 172.21.15.191:0/195009927 >> v1:172.21.15.138:6789/0 conn(0x55bbaa0ccc00 legacy=0x55bbaa1b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe64728700 1 -- 172.21.15.191:0/195009927 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f5d40 con 0x55bbaa0cc800 2024-04-27T04:36:29.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe64728700 1 --2- 172.21.15.191:0/195009927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x55bbaa3af9e0 tx=0x55bbaa1bd980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19907d0453bdccad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63726700 1 -- 172.21.15.191:0/195009927 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa1854a0 con 0x55bbaa0cc800 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63726700 1 -- 172.21.15.191:0/195009927 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa3f7680 con 0x55bbaa0cc800 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.473+0000 7fbe63726700 1 -- 172.21.15.191:0/195009927 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa185600 con 0x55bbaa0cc800 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 -- 172.21.15.191:0/195009927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 msgr2=0x55bbaa3b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/195009927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x55bbaa3af9e0 tx=0x55bbaa1bd980 comp rx=0 tx=0).stop 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 -- 172.21.15.191:0/195009927 shutdown_connections 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/195009927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 -- 172.21.15.191:0/195009927 >> 172.21.15.191:0/195009927 conn(0x55bbaa0cc400 msgr2=0x55bbaa1b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 -- 172.21.15.191:0/195009927 shutdown_connections 2024-04-27T04:36:29.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.474+0000 7fbe6677ea80 1 -- 172.21.15.191:0/195009927 wait complete. 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe6677ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe6677ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaa16e500 con 0x55bbaa0cc800 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe64728700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe64728700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:40994/0 (socket says 172.21.15.191:40994) 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe64728700 1 -- 172.21.15.191:0/2503991046 learned_addr learned my addr 172.21.15.191:0/2503991046 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe64728700 1 -- 172.21.15.191:0/2503991046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f5ba0 con 0x55bbaa0cc800 2024-04-27T04:36:29.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.475+0000 7fbe64728700 1 --2- 172.21.15.191:0/2503991046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55bbaae9d800 tx=0x55bbaa1bd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430ee7b682e2a9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe62f25700 1 -- 172.21.15.191:0/2503991046 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa185760 con 0x55bbaa0cc800 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe62f25700 1 -- 172.21.15.191:0/2503991046 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa3f7980 con 0x55bbaa0cc800 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe62f25700 1 -- 172.21.15.191:0/2503991046 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa1851e0 con 0x55bbaa0cc800 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- 172.21.15.191:0/2503991046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 msgr2=0x55bbaa3b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/2503991046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55bbaae9d800 tx=0x55bbaa1bd800 comp rx=0 tx=0).stop 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- 172.21.15.191:0/2503991046 shutdown_connections 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/2503991046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- 172.21.15.191:0/2503991046 >> 172.21.15.191:0/2503991046 conn(0x55bbaa0cc400 msgr2=0x55bbaa1b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- 172.21.15.191:0/2503991046 shutdown_connections 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- 172.21.15.191:0/2503991046 wait complete. 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.476+0000 7fbe6677ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaa16e3c0 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe64728700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe64728700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41002/0 (socket says 172.21.15.191:41002) 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe64728700 1 -- 172.21.15.191:0/3762881918 learned_addr learned my addr 172.21.15.191:0/3762881918 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe64728700 1 -- 172.21.15.191:0/3762881918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f5a00 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe64728700 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55bbaae9dec0 tx=0x55bbaa1bd980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe61722700 1 -- 172.21.15.191:0/3762881918 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa1854a0 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe61722700 1 -- 172.21.15.191:0/3762881918 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa3f7b00 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.477+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbaa0f5860 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe61722700 1 -- 172.21.15.191:0/3762881918 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa185600 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbaa0f56c0 con 0x55bbaa0cc800 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe6677ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe6677ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaa16e000 con 0x55bbaa0cd400 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe61722700 1 -- 172.21.15.191:0/3762881918 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbaae6eb00 con 0x55bbaa0cc800 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe61722700 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45a000 0x55bbaa3b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe61722700 1 -- 172.21.15.191:0/3762881918 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bbaa147180 con 0x55bbaa0cc800 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe64f29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.478+0000 7fbe64728700 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45a000 0x55bbaa3b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe64f29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41018/0 (socket says 172.21.15.191:41018) 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe64f29700 1 -- 172.21.15.191:0/3694951908 learned_addr learned my addr 172.21.15.191:0/3694951908 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe64728700 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45a000 0x55bbaa3b5080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55bbaaf62fc0 tx=0x55bbaa420b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe64f29700 1 -- 172.21.15.191:0/3694951908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f4000 con 0x55bbaa0cd400 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe64f29700 1 --2- 172.21.15.191:0/3694951908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55bbaaf623f0 tx=0x55bbaa420e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad982f520e3762f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe5ef1d700 1 -- 172.21.15.191:0/3694951908 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa1854a0 con 0x55bbaa0cd400 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe5ef1d700 1 -- 172.21.15.191:0/3694951908 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa16dc80 con 0x55bbaa0cd400 2024-04-27T04:36:29.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.479+0000 7fbe5ef1d700 1 -- 172.21.15.191:0/3694951908 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa185600 con 0x55bbaa0cd400 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3694951908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 msgr2=0x55bbaa3b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/3694951908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55bbaaf623f0 tx=0x55bbaa420e40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3694951908 shutdown_connections 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/3694951908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3694951908 >> 172.21.15.191:0/3694951908 conn(0x55bbaa0ccc00 msgr2=0x55bbaa1b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3694951908 shutdown_connections 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3694951908 wait complete. 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe6677ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaa108640 con 0x55bbaa0cd400 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe64f29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe64f29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41022/0 (socket says 172.21.15.191:41022) 2024-04-27T04:36:29.493 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.480+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 learned_addr learned my addr 172.21.15.191:0/1300114399 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f4ea0 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55bbaaf621b0 tx=0x55bbaa420a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe5d71a700 1 -- 172.21.15.191:0/1300114399 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa185760 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe5d71a700 1 -- 172.21.15.191:0/1300114399 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa16db00 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbaa0f4820 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe5d71a700 1 -- 172.21.15.191:0/1300114399 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa1854a0 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.481+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbaa0f49c0 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe5d71a700 1 -- 172.21.15.191:0/1300114399 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbaae6edc0 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe5d71a700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45ac00 0x55bbaa3b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe5d71a700 1 -- 172.21.15.191:0/1300114399 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bbaa147340 con 0x55bbaa0cd400 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe63f27700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45ac00 0x55bbaa3b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbaa0cd800 0x55bbaa3b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.494 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa0cdc00 con 0x55bbaa0cd800 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbaa0cd800 0x55bbaa3b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe63f27700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45ac00 0x55bbaa3b5600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55bbaae9c030 tx=0x55bbaa5080c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.482+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbaa0cd800 0x55bbaa3b5b80 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 realm 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b000 con 0x55bbaa0cd800 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b400 con 0x55bbaa0cd800 2024-04-27T04:36:29.495 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b800 con 0x55bbaa0cd800 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbaa45bc00 0x55bbaa3b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.483+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b000 con 0x55bbaa45bc00 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.484+0000 7fbe64728700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbaa45bc00 0x55bbaa3b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.484+0000 7fbe64728700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbaa45bc00 0x55bbaa3b6100 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.484+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.484+0000 7fbe6677ea80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.484+0000 7fbe6677ea80 20 searching for the correct realm 2024-04-27T04:36:29.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.484+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b400 con 0x55bbaa45bc00 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe64728700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbaa45b400 0x55bbaa3b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b000 con 0x55bbaa45b400 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe63f27700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbaa45b400 0x55bbaa3b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe63f27700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbaa45b400 0x55bbaa3b6c00 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe63f27700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbaa45b000 0x55bbaa3b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.485+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.486+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbaa45b000 0x55bbaa3b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.486+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbaa45b000 0x55bbaa3b7180 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.486+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.486+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.486+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.486+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b400 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.487+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.488+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b400 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.489+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.490+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b400 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.491+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b400 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b400 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.492+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45bc00 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b400 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe6677ea80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe6677ea80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe6677ea80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe6677ea80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.493+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa0cdc00 con 0x55bbaa0cd800 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e400 con 0x55bbaa0cd800 2024-04-27T04:36:29.517 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e800 con 0x55bbaa45bc00 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 zone default found 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 4 Realm: () 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 zonegroup default 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51ec00 con 0x55bbaa45b000 2024-04-27T04:36:29.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.494+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f000 con 0x55bbaa45bc00 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f400 con 0x55bbaa45bc00 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 started sync module instance, tier type = 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.495+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f800 con 0x55bbaa45b400 2024-04-27T04:36:29.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.496+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.496+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94264503220992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e800 con 0x55bbaa45b400 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.496+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f000 con 0x55bbaa0cd800 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.497+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94264503220992] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.497+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.497+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94264503222144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f400 con 0x55bbaa0cd800 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.497+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e400 con 0x55bbaa45bc00 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.498+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94264503222144] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.499+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.499+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94264503223296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51ec00 con 0x55bbaa45bc00 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.499+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa0cdc00 con 0x55bbaa45b400 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.500+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.500+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94264503224448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa45b800 con 0x55bbaa45b400 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.500+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa0cd800 2024-04-27T04:36:29.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.500+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94264503223296] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.501+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94264503224448] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.501+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.501+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94264503225600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f800 con 0x55bbaa0cd800 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.501+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e800 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.502+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.502+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94264503226752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51fc00 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.502+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f000 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.502+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94264503225600] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.503+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94264503226752] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.503+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.503+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94264503227904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f400 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.503+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e000 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94264503227904] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.521 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94264503229056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f800 con 0x55bbaa45b000 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=0 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=1 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=2 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=3 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=4 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=5 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.505+0000 7fbe6677ea80 20 add_watcher() i=6 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.506+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94264503229056] v33'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.506+0000 7fbe6677ea80 20 add_watcher() i=7 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.506+0000 7fbe6677ea80 2 all 8 watchers are set, enabling cache 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.506+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e800 con 0x55bbaa0cd800 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.506+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.506+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94264504786944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51fc00 con 0x55bbaa0cd800 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.507+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94264504786944] v33'236 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.507+0000 7fbe4b6f6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.507+0000 7fbe6677ea80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.507+0000 7fbe6677ea80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.507+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe6677ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe6677ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaa108780 con 0x55bbaa51f400 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe63f27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe63f27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41038/0 (socket says 172.21.15.191:41038) 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe63f27700 1 -- 172.21.15.191:0/27309885 learned_addr learned my addr 172.21.15.191:0/27309885 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe63f27700 1 -- 172.21.15.191:0/27309885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f4b60 con 0x55bbaa51f400 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.508+0000 7fbe63f27700 1 --2- 172.21.15.191:0/27309885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55bbaa4bc4e0 tx=0x55bbaa421f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab693d7266e0179d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe4aef5700 1 -- 172.21.15.191:0/27309885 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa53a000 con 0x55bbaa51f400 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe4aef5700 1 -- 172.21.15.191:0/27309885 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa532180 con 0x55bbaa51f400 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe4aef5700 1 -- 172.21.15.191:0/27309885 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa53a160 con 0x55bbaa51f400 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 -- 172.21.15.191:0/27309885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 msgr2=0x55bbaa3b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/27309885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55bbaa4bc4e0 tx=0x55bbaa421f80 comp rx=0 tx=0).stop 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 -- 172.21.15.191:0/27309885 shutdown_connections 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/27309885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 -- 172.21.15.191:0/27309885 >> 172.21.15.191:0/27309885 conn(0x55bbaa51f000 msgr2=0x55bbaa1ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 -- 172.21.15.191:0/27309885 shutdown_connections 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.509+0000 7fbe6677ea80 1 -- 172.21.15.191:0/27309885 wait complete. 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe6677ea80 1 Processor -- start 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe6677ea80 1 -- start start 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe6677ea80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe6677ea80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbaa16e000 con 0x55bbaa51f400 2024-04-27T04:36:29.524 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe63f27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe63f27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41048/0 (socket says 172.21.15.191:41048) 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe63f27700 1 -- 172.21.15.191:0/1404639873 learned_addr learned my addr 172.21.15.191:0/1404639873 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe63f27700 1 -- 172.21.15.191:0/1404639873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbaa0f4d00 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe63f27700 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55bbaa53d200 tx=0x55bbaa5084e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.510+0000 7fbe496f2700 1 -- 172.21.15.191:0/1404639873 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa53a2c0 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.511+0000 7fbe496f2700 1 -- 172.21.15.191:0/1404639873 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bbaa532300 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.511+0000 7fbe496f2700 1 -- 172.21.15.191:0/1404639873 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bbaa53a000 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.511+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bbaa0f4340 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.511+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbaa0f4b60 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.511+0000 7fbe496f2700 1 -- 172.21.15.191:0/1404639873 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bbaae6fb80 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.511+0000 7fbe496f2700 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa5c0000 0x55bbaa3b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.512+0000 7fbe496f2700 1 -- 172.21.15.191:0/1404639873 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bbaa147500 con 0x55bbaa51f400 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.512+0000 7fbe6677ea80 5 note: GC not initialized 2024-04-27T04:36:29.525 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.512+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa5c0000 0x55bbaa3b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.526 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.512+0000 7fbe64f29700 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa5c0000 0x55bbaa3b7700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55bbaa4bd800 tx=0x55bbaa508360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.512+0000 7fbe46eed700 20 reqs_thread_entry: start 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe6677ea80 20 init_complete bucket index max shards: 11 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe6677ea80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe44ee9700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:59 2024 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe446c7700 20 reqs_thread_entry: start 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe6677ea80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51f800 con 0x55bbaa0cd800 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe6677ea80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.513+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa51e800 con 0x55bbaa45b000 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.514+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 152+0+15 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.532 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.514+0000 7fbe4b6f6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.514+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94264504786944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c0400 con 0x55bbaa0cd800 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.515+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94264504786944] v33'237 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.515+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94264503220992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c0800 con 0x55bbaa45b400 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.516+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94264503220992] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.516+0000 7fbe6677ea80 20 remove_watcher() i=0 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.516+0000 7fbe6677ea80 2 removed watcher, disabling cache 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.516+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94264503222144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c0c00 con 0x55bbaa0cd800 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.518+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94264503222144] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.518+0000 7fbe6677ea80 20 remove_watcher() i=1 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.518+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94264503223296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c1000 con 0x55bbaa45bc00 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.519+0000 7fbe64728700 1 -- 172.21.15.191:0/1300114399 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94264503223296] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511200 con 0x55bbaa45bc00 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.519+0000 7fbe6677ea80 20 remove_watcher() i=2 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.519+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94264503224448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c1400 con 0x55bbaa45b400 2024-04-27T04:36:29.533 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.520+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94264503224448] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa511440 con 0x55bbaa45b400 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.520+0000 7fbe6677ea80 20 remove_watcher() i=3 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.520+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94264503225600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c1800 con 0x55bbaa0cd800 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.521+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94264503225600] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa0cd800 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.521+0000 7fbe6677ea80 20 remove_watcher() i=4 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.521+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94264503226752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c1c00 con 0x55bbaa45b000 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.522+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94264503226752] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.522+0000 7fbe6677ea80 20 remove_watcher() i=5 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.522+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94264503227904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c0400 con 0x55bbaa45b000 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.523+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94264503227904] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.523+0000 7fbe6677ea80 20 remove_watcher() i=6 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.523+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94264503229056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55bbaa5c0c00 con 0x55bbaa45b000 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.524+0000 7fbe64f29700 1 -- 172.21.15.191:0/1300114399 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94264503229056] v33'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bbaa510b40 con 0x55bbaa45b000 2024-04-27T04:36:29.534 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.524+0000 7fbe6677ea80 20 remove_watcher() i=7 2024-04-27T04:36:29.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbaa45b000 msgr2=0x55bbaa3b7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55bbaa45b000 0x55bbaa3b7180 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbaa0cd800 msgr2=0x55bbaa3b5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55bbaa0cd800 0x55bbaa3b5b80 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.535 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbaa45b400 msgr2=0x55bbaa3b6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55bbaa45b400 0x55bbaa3b6c00 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbaa45bc00 msgr2=0x55bbaa3b6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55bbaa45bc00 0x55bbaa3b6100 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.537 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45ac00 msgr2=0x55bbaa3b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45ac00 0x55bbaa3b5600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55bbaae9c030 tx=0x55bbaa5080c0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 msgr2=0x55bbaa3b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1300114399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cd400 0x55bbaa3b4b00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55bbaaf621b0 tx=0x55bbaa420a80 comp rx=0 tx=0).stop 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 reap_dead start 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 shutdown_connections 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe63f27700 1 -- 172.21.15.191:0/1300114399 reap_dead start 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 >> 172.21.15.191:0/1300114399 conn(0x55bbaa0ccc00 msgr2=0x55bbaa1b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 shutdown_connections 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1300114399 wait complete. 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa5c0000 msgr2=0x55bbaa3b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa5c0000 0x55bbaa3b7700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55bbaa4bd800 tx=0x55bbaa508360 comp rx=0 tx=0).stop 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 msgr2=0x55bbaa3b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55bbaa53d200 tx=0x55bbaa5084e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.538 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 shutdown_connections 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa5c0000 0x55bbaa3b7700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/1404639873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa51f400 0x55bbaa3b6680 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 >> 172.21.15.191:0/1404639873 conn(0x55bbaa51f000 msgr2=0x55bbaa1ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 shutdown_connections 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.525+0000 7fbe6677ea80 1 -- 172.21.15.191:0/1404639873 wait complete. 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45a000 msgr2=0x55bbaa3b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45a000 0x55bbaa3b5080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55bbaaf62fc0 tx=0x55bbaa420b40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 msgr2=0x55bbaa3b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55bbaae9dec0 tx=0x55bbaa1bd980 comp rx=0 tx=0).stop 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 shutdown_connections 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55bbaa45a000 0x55bbaa3b5080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 --2- 172.21.15.191:0/3762881918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbaa0cc800 0x55bbaa3b4580 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 >> 172.21.15.191:0/3762881918 conn(0x55bbaa0cc400 msgr2=0x55bbaa1b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.539 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 shutdown_connections 2024-04-27T04:36:29.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.526+0000 7fbe6677ea80 1 -- 172.21.15.191:0/3762881918 wait complete. 2024-04-27T04:36:29.540 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:29.540 INFO:teuthology.orchestra.run.smithi191.stdout: "entries": [], 2024-04-27T04:36:29.541 INFO:teuthology.orchestra.run.smithi191.stdout: "summary": [] 2024-04-27T04:36:29.541 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:29.541 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-27T04:36:29.541 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-27T04:36:29.541 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-27T04:36:29.541 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-04-27T04:36:29.587 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:29.588 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.586+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3464473a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55abb3c7cc00 0x55abb3f64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3464473a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55abb3fa7500 con 0x55abb3c7c800 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3464473a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55abb3d1ea00 con 0x55abb3c7cc00 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3461c1c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55abb3c7cc00 0x55abb3f64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f346241d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55abb3c7c800 0x55abb3d68800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46342/0 (socket says 172.21.15.191:46342) 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f346241d700 1 -- 172.21.15.191:0/4258772500 learned_addr learned my addr 172.21.15.191:0/4258772500 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:29.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3461c1c700 1 -- 172.21.15.191:0/4258772500 >> v1:172.21.15.138:6789/0 conn(0x55abb3c7c800 legacy=0x55abb3d68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f3461c1c700 1 -- 172.21.15.191:0/4258772500 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca5d40 con 0x55abb3c7cc00 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.587+0000 7f346141b700 1 -- 172.21.15.191:0/4258772500 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145839226 0 0) 0x55abb4a42000 con 0x55abb3c7c800 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3461c1c700 1 --2- 172.21.15.191:0/4258772500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7cc00 0x55abb3f64580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55abb3f5f890 tx=0x55abb3d6d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b746c6703203b735 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f346141b700 1 -- 172.21.15.191:0/4258772500 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d354a0 con 0x55abb3c7cc00 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f346141b700 1 -- 172.21.15.191:0/4258772500 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb3fa7800 con 0x55abb3c7cc00 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f346141b700 1 -- 172.21.15.191:0/4258772500 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d35600 con 0x55abb3c7cc00 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 -- 172.21.15.191:0/4258772500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7cc00 msgr2=0x55abb3f64580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 --2- 172.21.15.191:0/4258772500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7cc00 0x55abb3f64580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55abb3f5f890 tx=0x55abb3d6d920 comp rx=0 tx=0).stop 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 -- 172.21.15.191:0/4258772500 shutdown_connections 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 --2- 172.21.15.191:0/4258772500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7cc00 0x55abb3f64580 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 -- 172.21.15.191:0/4258772500 >> 172.21.15.191:0/4258772500 conn(0x55abb3c7c400 msgr2=0x55abb3d68000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 -- 172.21.15.191:0/4258772500 shutdown_connections 2024-04-27T04:36:29.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.588+0000 7f3464473a80 1 -- 172.21.15.191:0/4258772500 wait complete. 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f3464473a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f3464473a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb3d1e500 con 0x55abb3c7c800 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f346241d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f346241d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41066/0 (socket says 172.21.15.191:41066) 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.589+0000 7f346241d700 1 -- 172.21.15.191:0/3615862779 learned_addr learned my addr 172.21.15.191:0/3615862779 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.606 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f346241d700 1 -- 172.21.15.191:0/3615862779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca5ba0 con 0x55abb3c7c800 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f346241d700 1 --2- 172.21.15.191:0/3615862779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x55abb4a45770 tx=0x55abb3d6db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71e82130b641b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3460c1a700 1 -- 172.21.15.191:0/3615862779 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d351e0 con 0x55abb3c7c800 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3460c1a700 1 -- 172.21.15.191:0/3615862779 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb3fa7b00 con 0x55abb3c7c800 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3460c1a700 1 -- 172.21.15.191:0/3615862779 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d354a0 con 0x55abb3c7c800 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 -- 172.21.15.191:0/3615862779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 msgr2=0x55abb3f64580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 --2- 172.21.15.191:0/3615862779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x55abb4a45770 tx=0x55abb3d6db60 comp rx=0 tx=0).stop 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 -- 172.21.15.191:0/3615862779 shutdown_connections 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 --2- 172.21.15.191:0/3615862779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 -- 172.21.15.191:0/3615862779 >> 172.21.15.191:0/3615862779 conn(0x55abb3c7c400 msgr2=0x55abb3d69000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 -- 172.21.15.191:0/3615862779 shutdown_connections 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.590+0000 7f3464473a80 1 -- 172.21.15.191:0/3615862779 wait complete. 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f3464473a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f3464473a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb3d1e3c0 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f346241d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f346241d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41070/0 (socket says 172.21.15.191:41070) 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f346241d700 1 -- 172.21.15.191:0/2302229944 learned_addr learned my addr 172.21.15.191:0/2302229944 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f346241d700 1 -- 172.21.15.191:0/2302229944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca5a00 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f346241d700 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x55abb4a45e30 tx=0x55abb3d6da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f345f417700 1 -- 172.21.15.191:0/2302229944 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d35600 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f345f417700 1 -- 172.21.15.191:0/2302229944 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb3d1dc80 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f345f417700 1 -- 172.21.15.191:0/2302229944 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d351e0 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.591+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abb3ca5860 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abb3ca56c0 con 0x55abb3c7c800 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3464473a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3464473a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb3cb8640 con 0x55abb3c7d400 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f345f417700 1 -- 172.21.15.191:0/2302229944 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55abb4a1eb00 con 0x55abb3c7c800 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3462c1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f345f417700 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400a000 0x55abb3f65080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3462c1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41080/0 (socket says 172.21.15.191:41080) 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f3462c1e700 1 -- 172.21.15.191:0/722321895 learned_addr learned my addr 172.21.15.191:0/722321895 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.592+0000 7f345f417700 1 -- 172.21.15.191:0/2302229944 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55abb3cf7180 con 0x55abb3c7c800 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f346241d700 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400a000 0x55abb3f65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3462c1e700 1 -- 172.21.15.191:0/722321895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca4000 con 0x55abb3c7d400 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3462c1e700 1 --2- 172.21.15.191:0/722321895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55abb4b0ff50 tx=0x55abb3fc2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2ae83e774292f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f346241d700 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400a000 0x55abb3f65080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55abb4b0ef30 tx=0x55abb3fc2240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f345cc12700 1 -- 172.21.15.191:0/722321895 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d35600 con 0x55abb3c7d400 2024-04-27T04:36:29.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f345cc12700 1 -- 172.21.15.191:0/722321895 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb3fa6300 con 0x55abb3c7d400 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f345cc12700 1 -- 172.21.15.191:0/722321895 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d351e0 con 0x55abb3c7d400 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 -- 172.21.15.191:0/722321895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 msgr2=0x55abb3f64b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 --2- 172.21.15.191:0/722321895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55abb4b0ff50 tx=0x55abb3fc2de0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 -- 172.21.15.191:0/722321895 shutdown_connections 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 --2- 172.21.15.191:0/722321895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 -- 172.21.15.191:0/722321895 >> 172.21.15.191:0/722321895 conn(0x55abb3c7cc00 msgr2=0x55abb3d69800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 -- 172.21.15.191:0/722321895 shutdown_connections 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.593+0000 7f3464473a80 1 -- 172.21.15.191:0/722321895 wait complete. 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3464473a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3464473a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb3cb88c0 con 0x55abb3c7d400 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3462c1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3462c1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41096/0 (socket says 172.21.15.191:41096) 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 learned_addr learned my addr 172.21.15.191:0/3482203684 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca4ea0 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.594+0000 7f3462c1e700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55abb4b0e4e0 tx=0x55abb3fc2840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f345b40f700 1 -- 172.21.15.191:0/3482203684 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d35a20 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f345b40f700 1 -- 172.21.15.191:0/3482203684 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb3d1dc80 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abb3ca4820 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f345b40f700 1 -- 172.21.15.191:0/3482203684 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d35600 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abb3ca49c0 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f345b40f700 1 -- 172.21.15.191:0/3482203684 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55abb4a1edc0 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.595+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f345b40f700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400ac00 0x55abb3f65600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f345b40f700 1 -- 172.21.15.191:0/3482203684 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55abb3cf7340 con 0x55abb3c7d400 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3461c1c700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400ac00 0x55abb3f65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb3c7d800 0x55abb3f65b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb3c7d800 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3462c1e700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb3c7d800 0x55abb3f65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3461c1c700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400ac00 0x55abb3f65600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55abb3f5f890 tx=0x55abb3d6d800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3462c1e700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb3c7d800 0x55abb3f65b80 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3464473a80 20 realm 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.596+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b000 con 0x55abb3c7d800 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:36:29.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b400 con 0x55abb3c7d800 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b800 con 0x55abb3c7d800 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb400bc00 0x55abb3f66100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b000 con 0x55abb400bc00 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f346241d700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb400bc00 0x55abb3f66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.597+0000 7f346241d700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb400bc00 0x55abb3f66100 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f3464473a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f3464473a80 20 searching for the correct realm 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b400 con 0x55abb400bc00 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f346241d700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb400b400 0x55abb3f66c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b000 con 0x55abb400b400 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f3461c1c700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb400b400 0x55abb3f66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.598+0000 7f3461c1c700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb400b400 0x55abb3f66c00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3461c1c700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb400b000 0x55abb3f67180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3462c1e700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb400b000 0x55abb3f67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3462c1e700 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb400b000 0x55abb3f67180 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.599+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.600+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b400 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.601+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b400 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.602+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.603+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b400 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.604+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b400 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b400 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.605+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400bc00 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b400 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3464473a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3464473a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3464473a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3464473a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.606+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb3c7d800 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb3c7d800 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8800 con 0x55abb400bc00 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 zone default found 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 4 Realm: () 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 zonegroup default 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.607+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb400b000 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9000 con 0x55abb400bc00 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9400 con 0x55abb400bc00 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 started sync module instance, tier type = 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.608+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb400b400 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.609+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.609+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94195946994432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8800 con 0x55abb400b400 2024-04-27T04:36:29.633 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.609+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9000 con 0x55abb3c7d800 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.610+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94195946994432] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.611+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.611+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94195946995584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9400 con 0x55abb3c7d800 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.611+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb400bc00 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.612+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94195946995584] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.612+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.612+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94195946996736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb400bc00 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.612+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb400b400 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.613+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94195946996736] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.613+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.613+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94195946997888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb400b800 con 0x55abb400b400 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.613+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb3c7d800 2024-04-27T04:36:29.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.616+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.616+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94195946999040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb3c7d800 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.616+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8800 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.616+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94195946997888] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.617+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94195946999040] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.617+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.617+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94195947000192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.617+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9000 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.618+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94195947000192] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.618+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.618+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94195947001344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9400 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.618+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8000 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94195947001344] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94195947002496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb400b000 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=0 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=1 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=2 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=3 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=4 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=5 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.619+0000 7f3464473a80 20 add_watcher() i=6 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.620+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94195947002496] v33'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.620+0000 7f3464473a80 20 add_watcher() i=7 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.620+0000 7f3464473a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.620+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8800 con 0x55abb3c7d800 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.621+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.621+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94195948584960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb3c7d800 2024-04-27T04:36:29.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94195948584960] v33'238 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f34493eb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3464473a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3464473a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3464473a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.622+0000 7f3464473a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb3d1e3c0 con 0x55abb40d9400 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3461c1c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3461c1c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41102/0 (socket says 172.21.15.191:41102) 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3461c1c700 1 -- 172.21.15.191:0/2857733172 learned_addr learned my addr 172.21.15.191:0/2857733172 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3461c1c700 1 -- 172.21.15.191:0/2857733172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca4b60 con 0x55abb40d9400 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3461c1c700 1 --2- 172.21.15.191:0/2857733172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55abb4009440 tx=0x55abb3d6d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad5005a80c1294e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3448bea700 1 -- 172.21.15.191:0/2857733172 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb3d35760 con 0x55abb40d9400 2024-04-27T04:36:29.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3448bea700 1 -- 172.21.15.191:0/2857733172 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb40f8000 con 0x55abb40d9400 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.623+0000 7f3448bea700 1 -- 172.21.15.191:0/2857733172 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb40fa000 con 0x55abb40d9400 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- 172.21.15.191:0/2857733172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 msgr2=0x55abb3f66680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 --2- 172.21.15.191:0/2857733172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55abb4009440 tx=0x55abb3d6d980 comp rx=0 tx=0).stop 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- 172.21.15.191:0/2857733172 shutdown_connections 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 --2- 172.21.15.191:0/2857733172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- 172.21.15.191:0/2857733172 >> 172.21.15.191:0/2857733172 conn(0x55abb40d9000 msgr2=0x55abb3d6a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- 172.21.15.191:0/2857733172 shutdown_connections 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- 172.21.15.191:0/2857733172 wait complete. 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 Processor -- start 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- start start 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.624+0000 7f3464473a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb3d1e500 con 0x55abb40d9400 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3461c1c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3461c1c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41118/0 (socket says 172.21.15.191:41118) 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 learned_addr learned my addr 172.21.15.191:0/1332387098 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb3ca4d00 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3461c1c700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x55abb40ed260 tx=0x55abb40b68a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f34473e7700 1 -- 172.21.15.191:0/1332387098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb40fa160 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f34473e7700 1 -- 172.21.15.191:0/1332387098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abb40f8180 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abb3ca4340 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f34473e7700 1 -- 172.21.15.191:0/1332387098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55abb40fa2c0 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.625+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abb3ca5d40 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f34473e7700 1 -- 172.21.15.191:0/1332387098 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55abb4a1fb80 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f34473e7700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb4172000 0x55abb3f67700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f34473e7700 1 -- 172.21.15.191:0/1332387098 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55abb3cf7500 con 0x55abb40d9400 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f3462c1e700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb4172000 0x55abb3f67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb40d8000 0x55abb4184000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb40d8000 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.626+0000 7f346241d700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb40d8000 0x55abb4184000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.627+0000 7f3462c1e700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb4172000 0x55abb3f67700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55abb406c8d0 tx=0x55abb40b65a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.627+0000 7f346241d700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb40d8000 0x55abb4184000 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.628+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'61 uv54 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb40d8000 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.628+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb40d8800 0x55abb4184580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.628+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4172400 con 0x55abb40d8800 2024-04-27T04:36:29.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.628+0000 7f3461c1c700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb40d8800 0x55abb4184580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.628+0000 7f3461c1c700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb40d8800 0x55abb4184580 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.629+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.630+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb4172800 0x55abb4184b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.630+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4172c00 con 0x55abb4172800 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.630+0000 7f3462c1e700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb4172800 0x55abb4184b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.630+0000 7f3462c1e700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb4172800 0x55abb4184b00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.631+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.631+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb40d8000 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.632+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'239 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb40d8000 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.632+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb4173400 0x55abb4185080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.632+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173800 con 0x55abb4173400 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.632+0000 7f346241d700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb4173400 0x55abb4185080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.632+0000 7f346241d700 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb4173400 0x55abb4185080 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.633+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'83 uv60 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.633+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173c00 con 0x55abb4172800 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.634+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'62 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.634+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb40d8000 2024-04-27T04:36:29.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.635+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb40d8000 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.635+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb40d8800 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.636+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'57 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.636+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb4173400 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.637+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.637+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb4172800 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.638+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.638+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6000 con 0x55abb4173400 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.639+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'84 uv62 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.639+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6400 con 0x55abb40d8800 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.640+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'36 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.640+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6800 con 0x55abb4173400 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.641+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'40 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.641+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6c00 con 0x55abb40d8800 2024-04-27T04:36:29.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.642+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'72 uv56 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.642+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7000 con 0x55abb40d8000 2024-04-27T04:36:29.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.644+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'240 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb40d8000 2024-04-27T04:36:29.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.644+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7400 con 0x55abb4172800 2024-04-27T04:36:29.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.645+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'74 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.645+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7800 con 0x55abb4173400 2024-04-27T04:36:29.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.646+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'85 uv64 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.646+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7c00 con 0x55abb4172800 2024-04-27T04:36:29.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.647+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'40 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.647+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6000 con 0x55abb40d8000 2024-04-27T04:36:29.649 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.648+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'37 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb40d8000 2024-04-27T04:36:29.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.648+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6800 con 0x55abb40d8000 2024-04-27T04:36:29.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.649+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'241 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb40d8000 2024-04-27T04:36:29.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.649+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6400 con 0x55abb4172800 2024-04-27T04:36:29.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.650+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'75 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.651 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.650+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb4173400 2024-04-27T04:36:29.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.651+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'44 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.652 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.651+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb4172800 2024-04-27T04:36:29.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.652+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.653 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.652+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173800 con 0x55abb4172800 2024-04-27T04:36:29.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.653+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'46 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.654 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.653+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb4172800 2024-04-27T04:36:29.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.653+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'31 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.654+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4172400 con 0x55abb4172800 2024-04-27T04:36:29.656 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.654+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'63 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.656 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.654+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb40d8800 2024-04-27T04:36:29.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.656+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.657 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.656+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7400 con 0x55abb4172800 2024-04-27T04:36:29.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.656+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'41 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.658 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.656+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7c00 con 0x55abb40d8800 2024-04-27T04:36:29.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.658+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'58 uv44 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.659 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.658+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6400 con 0x55abb4172800 2024-04-27T04:36:29.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.659+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'76 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.660 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.659+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb40d8800 2024-04-27T04:36:29.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.660+0000 7f3461c1c700 1 -- 172.21.15.191:0/1332387098 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'73 uv58 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb40d8800 2024-04-27T04:36:29.661 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.660+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb4173400 2024-04-27T04:36:29.662 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.661+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb4173400 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3444be2700 20 reqs_thread_entry: start 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 20 init_complete bucket index max shards: 11 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3442bde700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:37:00 2024 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f34423bc700 20 reqs_thread_entry: start 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173c00 con 0x55abb3c7d800 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T04:36:29.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.662+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb400b400 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv43 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 : miss 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:64 8.17 8:ec0d9c72:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb3c7d800 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(64 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 info.flags=0x17 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 10 adding default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 to cache LRU end 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:36:29.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:65 4.19 4:9b384b3a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7000 con 0x55abb400bc00 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.663+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(65 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.664+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:66 4.37 4:ee7fb757:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7800 con 0x55abb400b000 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.664+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(66 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.664+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 4.36 4:6c76ca4e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6000 con 0x55abb3c7d800 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(67 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 4.5 4:a1c9c557:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6800 con 0x55abb400bc00 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(68 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:69 4.1b 4:d853d9f1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb400b400 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(69 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 [call out=425b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f3464473a80 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.665+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 4.1e 4:780d6fff:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb400b000 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.666+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.666+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 4.33 4:cf60569a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb400b000 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.666+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.666+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:72 4.b 4:d0df9453:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb400b400 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.666+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 14 ==== osd_op_reply(72 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.666+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:73 4.2 4:43ef8a25:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7400 con 0x55abb3c7d800 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(73 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 4.25 4:a6c40cd6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6400 con 0x55abb400b400 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 15 ==== osd_op_reply(74 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:75 4.29 4:952fbabd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6c00 con 0x55abb400bc00 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== osd_op_reply(75 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f3464473a80 20 get_obj_state: rctx=0x55abb40b1a40 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55abb422e1e8 s->prefetch_data=0 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.667+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:76 3.15s0 3:ab64c295:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7c00 con 0x55abb400b400 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 16 ==== osd_op_reply(76 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=877b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3464473a80 10 manifest: total_size = 6 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3464473a80 20 get_obj_state: setting s->obj_tag to e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4221.6895876833921228214 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3464473a80 20 get_obj_state: rctx=0x55abb40b1a40 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55abb422e1e8 s->prefetch_data=0 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3464473a80 20 prepare_atomic_modification: state is not atomic. state=0x55abb422e1e8 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3464473a80 20 bucket index object: default.rgw.buckets.index:.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.668+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:77 4.1b 4:d853d9f1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb400b400 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.669+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 17 ==== osd_op_reply(77 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 [stat,call,call] v33'8 uv8 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.675 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.669+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:78 3.15s0 3:ab64c295:::e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173800 con 0x55abb400b400 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 18 ==== osd_op_reply(78 e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6_15f6f8dc036519d7fe15b39338f6e5db [call] v33'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:79 4.1b 4:d853d9f1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4172400 con 0x55abb400b400 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 20 list_objects_ordered: starting attempt 1 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:80 4.19 4:9b384b3a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173c00 con 0x55abb400bc00 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:81 4.37 4:ee7fb757:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173800 con 0x55abb400b000 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:82 4.36 4:6c76ca4e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb3c7d800 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:83 4.5 4:a1c9c557:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6400 con 0x55abb400bc00 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:84 4.1b 4:d853d9f1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7c00 con 0x55abb400b400 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:85 4.1e 4:780d6fff:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb400b000 2024-04-27T04:36:29.682 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:86 4.33 4:cf60569a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb400b000 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:87 4.b 4:d0df9453:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac000 con 0x55abb400b400 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:88 4.2 4:43ef8a25:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac400 con 0x55abb3c7d800 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:89 4.25 4:a6c40cd6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac800 con 0x55abb400b400 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.670+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:90 4.29 4:952fbabd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42acc00 con 0x55abb400bc00 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(90 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(81 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(80 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(86 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 21 ==== osd_op_reply(85 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(82 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 21 ==== osd_op_reply(83 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 19 ==== osd_op_reply(89 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 20 ==== osd_op_reply(87 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.671+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(88 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 21 ==== osd_op_reply(79 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 [stat,call,call] v33'9 uv9 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f345c411700 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 22 ==== osd_op_reply(84 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 [call out=124b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6])"; uploads.size()=0; is_truncated=0 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6]) 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:91 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ad000 con 0x55abb400b400 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 23 ==== osd_op_reply(91 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv41 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 lifecycle: adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.673+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:92 8.17 8:ec0d9c72:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ad800 con 0x55abb3c7d800 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(92 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 [delete,create,call,call,writefull 0~352,setxattr (123)] v33'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 info.flags=0x17 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 to cache LRU end 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 10 lifecycle: distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6, ofs0, ns] 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:93 7.e 7:7759931f:::notify.3:head [notify cookie 94195948588416 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb400b400 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 24 ==== watch-notify(notify (1) cookie 94195946997888 notify 141733920801 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55abb4139ba0 con 0x55abb400b400 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f345c411700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920801 cookie 94195946997888 notifier 4852 bl.length()=725 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 25 ==== osd_op_reply(93 notify.3 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f345c411700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 info.flags=0x17 2024-04-27T04:36:29.685 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f345c411700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 to cache LRU end 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f345c411700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f345c411700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:94 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb400b400 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 26 ==== osd_op_reply(94 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 27 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920801 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb4139d40 con 0x55abb400b400 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6]): orig_sources= new_sources= 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6]): potential sources added=[] removed=[] 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6]): orig_dests= new_dests= 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 20 lifecycle: handle_bi_update(): bucket=:myfoo[e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6]): potential dests added=[] removed=[] 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.675+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:33 6.16 6:6d4431ed:lc::lc.7:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb4172800 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.676+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(33 lc.7 [call] v33'41 uv41 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.676+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 6.16 6:6d4431ed:lc::lc.7:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb4172800 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.677+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(34 lc.7 [call] v33'42 uv42 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.686 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.677+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:35 6.16 6:6d4431ed:lc::lc.7:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb4172800 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3462c1e700 1 -- 172.21.15.191:0/1332387098 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(35 lc.7 [call] v33'43 uv43 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb4172800 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:95 4.19 4:9b384b3a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb400bc00 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:96 4.37 4:ee7fb757:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173800 con 0x55abb400b000 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:97 4.36 4:6c76ca4e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7800 con 0x55abb3c7d800 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:98 4.5 4:a1c9c557:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6000 con 0x55abb400bc00 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:99 4.1b 4:d853d9f1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7400 con 0x55abb400b400 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:100 4.1e 4:780d6fff:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173c00 con 0x55abb400b000 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:101 4.33 4:cf60569a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4172400 con 0x55abb400b000 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:102 4.b 4:d0df9453:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb400b400 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:103 4.2 4:43ef8a25:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb3c7d800 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:104 4.25 4:a6c40cd6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42acc00 con 0x55abb400b400 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.678+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:105 4.29 4:952fbabd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac000 con 0x55abb400bc00 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 22 ==== osd_op_reply(98 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.687 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 23 ==== osd_op_reply(95 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(97 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(103 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 22 ==== osd_op_reply(96 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 23 ==== osd_op_reply(101 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 28 ==== osd_op_reply(99 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 [call out=124b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 24 ==== osd_op_reply(105 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 24 ==== osd_op_reply(100 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 29 ==== osd_op_reply(104 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 30 ==== osd_op_reply(102 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.679+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:106 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac800 con 0x55abb400b400 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.680+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 31 ==== osd_op_reply(106 foo.buckets [call] v33'50 uv50 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.680+0000 7f3464473a80 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2024-04-27T04:36:29.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.680+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:107 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ad000 con 0x55abb400b400 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 32 ==== osd_op_reply(107 myfoo [call,call,delete] v33'51 uv51 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f3464473a80 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f3464473a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94195948588416 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7000 con 0x55abb400b000 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 25 ==== watch-notify(notify (1) cookie 94195947001344 notify 141733920810 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x55abb4137a00 con 0x55abb400b000 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 26 ==== osd_op_reply(108 notify.6 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f345cc12700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920810 cookie 94195947001344 notifier 4852 bl.length()=149 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.681+0000 7f345cc12700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac400 con 0x55abb400b000 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.682+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 27 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920810 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb4135040 con 0x55abb400b000 2024-04-27T04:36:29.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.682+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 28 ==== osd_op_reply(109 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.682+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:110 8.17 8:ec0d9c72:root::.bucket.meta.myfoo%3ae734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6800 con 0x55abb3c7d800 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 30 ==== osd_op_reply(110 .bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 [call,call,delete] v33'3 uv3 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 10 removing default.rgw.meta+root+.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 from cache 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6, ofs0, ns] 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94195948588416 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6c00 con 0x55abb400b400 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 33 ==== watch-notify(notify (1) cookie 94195946997888 notify 141733920802 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x55abb4139d40 con 0x55abb400b400 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f345cc12700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920802 cookie 94195946997888 notifier 4852 bl.length()=206 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f345cc12700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42adc00 con 0x55abb400b400 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 34 ==== osd_op_reply(111 notify.3 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 35 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 36 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920802 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb4138340 con 0x55abb400b400 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:113 4.19 4:9b384b3a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6400 con 0x55abb400bc00 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:114 4.37 4:ee7fb757:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7c00 con 0x55abb400b000 2024-04-27T04:36:29.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:115 4.36 4:6c76ca4e:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb3c7d800 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:116 4.5 4:a1c9c557:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173800 con 0x55abb400bc00 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:117 4.1b 4:d853d9f1:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173c00 con 0x55abb400b400 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:118 4.1e 4:780d6fff:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7800 con 0x55abb400b000 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:119 4.33 4:cf60569a:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb400b000 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:120 4.b 4:d0df9453:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8400 con 0x55abb400b400 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:121 4.2 4:43ef8a25:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb3c7d800 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:122 4.25 4:a6c40cd6:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ad400 con 0x55abb400b400 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.683+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:123 4.29 4:952fbabd:::.dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ad800 con 0x55abb400bc00 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.685+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 37 ==== osd_op_reply(122 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.9 [delete] v33'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.685+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 25 ==== osd_op_reply(116 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.3 [delete] v33'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.685+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 26 ==== osd_op_reply(123 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.10 [delete] v33'8 uv8 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.685+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 29 ==== osd_op_reply(114 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.1 [delete] v33'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.685+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 31 ==== osd_op_reply(121 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.8 [delete] v33'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.685+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 32 ==== osd_op_reply(115 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.2 [delete] v33'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.686+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 30 ==== osd_op_reply(119 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.6 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.686+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 31 ==== osd_op_reply(118 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.5 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.687+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 38 ==== osd_op_reply(120 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.7 [delete] v33'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.687+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 39 ==== osd_op_reply(117 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.4 [delete] v33'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 27 ==== osd_op_reply(113 .dir.e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6.0 [delete] v33'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 : miss 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:124 6.18 6:1a1f1044:::pubsub..bucket.myfoo%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb400b000 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 32 ==== osd_op_reply(124 pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 info.flags=0x0 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 10 adding default.rgw.log++pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 to cache LRU end 2024-04-27T04:36:29.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.688+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:125 6.18 6:1a1f1044:::pubsub..bucket.myfoo%2fe734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173c00 con 0x55abb400b000 2024-04-27T04:36:29.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.690+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 33 ==== osd_op_reply(125 pubsub..bucket.myfoo/e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.6 [delete] v33'35 uv27 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.690+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:126 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42acc00 con 0x55abb400b400 2024-04-27T04:36:29.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.691+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 40 ==== osd_op_reply(126 foo.buckets [call] v33'52 uv52 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.691+0000 7f3464473a80 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-04-27T04:36:29.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.691+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:127 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ac800 con 0x55abb400b000 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.692+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 34 ==== osd_op_reply(127 9te6NH5mcdcq0Tc5i8i1 [delete] v33'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.692+0000 7f3464473a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.692+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:128 7.e 7:7759931f:::notify.3:head [notify cookie 94195948588416 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42ad000 con 0x55abb400b400 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 41 ==== watch-notify(notify (1) cookie 94195946997888 notify 141733920803 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55abb4138340 con 0x55abb400b400 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 42 ==== osd_op_reply(128 notify.3 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f345cc12700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920803 cookie 94195946997888 notifier 4852 bl.length()=170 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f345cc12700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:129 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7000 con 0x55abb400b400 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 43 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920803 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb41384e0 con 0x55abb400b400 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 44 ==== osd_op_reply(129 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.701 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f3464473a80 10 removing email index: foo@foo.com 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.693+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:130 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb42adc00 con 0x55abb3c7d800 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 33 ==== osd_op_reply(130 foo@foo.com [delete] v33'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f3464473a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:131 7.e 7:7759931f:::notify.3:head [notify cookie 94195948588416 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7400 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 45 ==== watch-notify(notify (1) cookie 94195946997888 notify 141733920804 ret 0) v3 ==== 204+0+0 (crc 0 0 0) 0x55abb41384e0 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f345cc12700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920804 cookie 94195946997888 notifier 4852 bl.length()=162 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 46 ==== osd_op_reply(131 notify.3 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.694+0000 7f345cc12700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:132 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb3c7dc00 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.695+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 47 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920804 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb4138680 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.695+0000 7f3464473a80 10 removing user buckets index 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.695+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 48 ==== osd_op_reply(132 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.695+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:133 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6c00 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 49 ==== osd_op_reply(133 foo.buckets [delete] v33'53 uv53 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.702 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3464473a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94195948588416 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4172400 con 0x55abb400b000 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 35 ==== watch-notify(notify (1) cookie 94195947000192 notify 141733920811 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55abb4135040 con 0x55abb400b000 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f345c411700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920811 cookie 94195947000192 notifier 4852 bl.length()=160 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 36 ==== osd_op_reply(134 notify.5 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f345c411700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4250000 con 0x55abb400b000 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 37 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920811 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb41351e0 con 0x55abb400b000 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 38 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3464473a80 10 removing user index: foo 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.696+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:136 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4173000 con 0x55abb3c7d800 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.697+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 34 ==== osd_op_reply(136 foo [call,call,delete] v33'20 uv20 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.697+0000 7f3464473a80 10 removing default.rgw.meta+users.uid+foo from cache 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.697+0000 7f3464473a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.697+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:137 7.d 7:b4812045:::notify.4:head [notify cookie 94195948588416 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9800 con 0x55abb3c7d800 2024-04-27T04:36:29.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.697+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 35 ==== watch-notify(notify (1) cookie 94195946999040 notify 141733920786 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55abb41351e0 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.697+0000 7f345c411700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920786 cookie 94195946999040 notifier 4852 bl.length()=152 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.698+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 36 ==== osd_op_reply(137 notify.4 [notify cookie 94195948588416 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.698+0000 7f345c411700 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4250400 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.698+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 37 ==== osd_op_reply(138 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.698+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 38 ==== watch-notify(notify_complete (2) cookie 94195948588416 notify 141733920786 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55abb4135380 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.698+0000 7f34493eb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.698+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:139 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94195948584960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d9c00 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.699+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 39 ==== osd_op_reply(139 data_loggenerations_metadata [watch unwatch cookie 94195948584960] v33'242 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.699+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:140 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94195946994432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d8c00 con 0x55abb400b400 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.700+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 50 ==== osd_op_reply(140 notify.0 [watch unwatch cookie 94195946994432] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.700+0000 7f3464473a80 20 remove_watcher() i=0 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.700+0000 7f3464473a80 2 removed watcher, disabling cache 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.700+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94195946995584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d6800 con 0x55abb3c7d800 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.702+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 40 ==== osd_op_reply(141 notify.1 [watch unwatch cookie 94195946995584] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.702+0000 7f3464473a80 20 remove_watcher() i=1 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.702+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:142 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94195946996736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7c00 con 0x55abb400bc00 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.703+0000 7f346241d700 1 -- 172.21.15.191:0/3482203684 <== osd.3 v2:172.21.15.191:6800/41887 28 ==== osd_op_reply(142 notify.2 [watch unwatch cookie 94195946996736] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1200 con 0x55abb400bc00 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.703+0000 7f3464473a80 20 remove_watcher() i=2 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.703+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:143 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94195946997888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb40d7800 con 0x55abb400b400 2024-04-27T04:36:29.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.704+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 <== osd.2 v2:172.21.15.191:6816/41888 51 ==== osd_op_reply(143 notify.3 [watch unwatch cookie 94195946997888] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c1440 con 0x55abb400b400 2024-04-27T04:36:29.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.704+0000 7f3464473a80 20 remove_watcher() i=3 2024-04-27T04:36:29.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.704+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:144 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94195946999040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4250c00 con 0x55abb3c7d800 2024-04-27T04:36:29.707 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.705+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.1 v2:172.21.15.191:6808/41886 41 ==== osd_op_reply(144 notify.4 [watch unwatch cookie 94195946999040] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb3c7d800 2024-04-27T04:36:29.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.705+0000 7f3464473a80 20 remove_watcher() i=4 2024-04-27T04:36:29.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.705+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:145 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94195947000192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4251000 con 0x55abb400b000 2024-04-27T04:36:29.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.706+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 39 ==== osd_op_reply(145 notify.5 [watch unwatch cookie 94195947000192] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.706+0000 7f3464473a80 20 remove_watcher() i=5 2024-04-27T04:36:29.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.706+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:146 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94195947001344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4251400 con 0x55abb400b000 2024-04-27T04:36:29.708 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.707+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 40 ==== osd_op_reply(146 notify.6 [watch unwatch cookie 94195947001344] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.707+0000 7f3464473a80 20 remove_watcher() i=6 2024-04-27T04:36:29.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.707+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:147 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94195947002496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55abb4251800 con 0x55abb400b000 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3462c1e700 1 -- 172.21.15.191:0/3482203684 <== osd.0 v2:172.21.15.138:6800/39496 41 ==== osd_op_reply(147 notify.7 [watch unwatch cookie 94195947002496] v33'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55abb40c0b40 con 0x55abb400b000 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 20 remove_watcher() i=7 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb400b000 msgr2=0x55abb3f67180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb400b000 0x55abb3f67180 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb3c7d800 msgr2=0x55abb3f65b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb3c7d800 0x55abb3f65b80 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb400b400 msgr2=0x55abb3f66c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb400b400 0x55abb3f66c00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb400bc00 msgr2=0x55abb3f66100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb400bc00 0x55abb3f66100 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400ac00 msgr2=0x55abb3f65600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400ac00 0x55abb3f65600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55abb3f5f890 tx=0x55abb3d6d800 comp rx=0 tx=0).stop 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 msgr2=0x55abb3f64b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/3482203684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7d400 0x55abb3f64b00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55abb4b0e4e0 tx=0x55abb3fc2840 comp rx=0 tx=0).stop 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 reap_dead start 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3461c1c700 1 -- 172.21.15.191:0/3482203684 reap_dead start 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 shutdown_connections 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 >> 172.21.15.191:0/3482203684 conn(0x55abb3c7cc00 msgr2=0x55abb3d69800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 shutdown_connections 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/3482203684 wait complete. 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb40d8800 msgr2=0x55abb4184580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55abb40d8800 0x55abb4184580 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb40d8000 msgr2=0x55abb4184000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55abb40d8000 0x55abb4184000 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.713 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb4173400 msgr2=0x55abb4185080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55abb4173400 0x55abb4185080 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb4172800 msgr2=0x55abb4184b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.708+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55abb4172800 0x55abb4184b00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb4172000 msgr2=0x55abb3f67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb4172000 0x55abb3f67700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55abb406c8d0 tx=0x55abb40b65a0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 msgr2=0x55abb3f66680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f346241d700 1 -- 172.21.15.191:0/1332387098 reap_dead start 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x55abb40ed260 tx=0x55abb40b68a0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 shutdown_connections 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/1332387098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb40d9400 0x55abb3f66680 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 >> 172.21.15.191:0/1332387098 conn(0x55abb40d9000 msgr2=0x55abb3d6a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 shutdown_connections 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/1332387098 wait complete. 2024-04-27T04:36:29.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400a000 msgr2=0x55abb3f65080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400a000 0x55abb3f65080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55abb4b0ef30 tx=0x55abb3fc2240 comp rx=0 tx=0).stop 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 msgr2=0x55abb3f64580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x55abb4a45e30 tx=0x55abb3d6da40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 shutdown_connections 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55abb400a000 0x55abb3f65080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 --2- 172.21.15.191:0/2302229944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abb3c7c800 0x55abb3f64580 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 >> 172.21.15.191:0/2302229944 conn(0x55abb3c7c400 msgr2=0x55abb3d69000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 shutdown_connections 2024-04-27T04:36:29.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.709+0000 7f3464473a80 1 -- 172.21.15.191:0/2302229944 wait complete. 2024-04-27T04:36:29.716 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T04:36:29.716 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T04:36:29.716 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T04:36:29.762 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:29.762 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bef19a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bef19a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d686f1b500 con 0x55d686bf0c00 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bef19a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d686c92a00 con 0x55d686bf0800 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bcec3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bc6c2700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55d686bf0c00 0x55d686cdc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46348/0 (socket says 172.21.15.191:46348) 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.761+0000 7f93bc6c2700 1 -- 172.21.15.191:0/685525851 learned_addr learned my addr 172.21.15.191:0/685525851 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bbec1700 1 -- 172.21.15.191:0/685525851 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856038784 0 0) 0x55d6879be000 con 0x55d686bf0c00 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bbec1700 1 -- 172.21.15.191:0/685525851 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55d686f1b680 con 0x55d686bf0c00 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bcec3700 1 -- 172.21.15.191:0/685525851 >> v1:172.21.15.138:6789/0 conn(0x55d686bf0c00 legacy=0x55d686cdc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.771 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bcec3700 1 -- 172.21.15.191:0/685525851 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c19d40 con 0x55d686bf0800 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bcec3700 1 --2- 172.21.15.191:0/685525851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55d686ed39e0 tx=0x55d686ce1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69959fba1626d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bbec1700 1 -- 172.21.15.191:0/685525851 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca94a0 con 0x55d686bf0800 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bbec1700 1 -- 172.21.15.191:0/685525851 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d686f1b680 con 0x55d686bf0800 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bbec1700 1 -- 172.21.15.191:0/685525851 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca9600 con 0x55d686bf0800 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 -- 172.21.15.191:0/685525851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 msgr2=0x55d686ed8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 --2- 172.21.15.191:0/685525851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55d686ed39e0 tx=0x55d686ce1980 comp rx=0 tx=0).stop 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 -- 172.21.15.191:0/685525851 shutdown_connections 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 --2- 172.21.15.191:0/685525851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 -- 172.21.15.191:0/685525851 >> 172.21.15.191:0/685525851 conn(0x55d686bf0400 msgr2=0x55d686cdc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 -- 172.21.15.191:0/685525851 shutdown_connections 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.762+0000 7f93bef19a80 1 -- 172.21.15.191:0/685525851 wait complete. 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bef19a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bef19a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d686c92500 con 0x55d686bf0800 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bcec3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bcec3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41134/0 (socket says 172.21.15.191:41134) 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.763+0000 7f93bcec3700 1 -- 172.21.15.191:0/4164840161 learned_addr learned my addr 172.21.15.191:0/4164840161 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bcec3700 1 -- 172.21.15.191:0/4164840161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c19ba0 con 0x55d686bf0800 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bcec3700 1 --2- 172.21.15.191:0/4164840161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x55d6879c1800 tx=0x55d686ce1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e95a368ca1c081 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bb6c0700 1 -- 172.21.15.191:0/4164840161 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca9760 con 0x55d686bf0800 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bb6c0700 1 -- 172.21.15.191:0/4164840161 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d686f1b980 con 0x55d686bf0800 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bb6c0700 1 -- 172.21.15.191:0/4164840161 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca94a0 con 0x55d686bf0800 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 -- 172.21.15.191:0/4164840161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 msgr2=0x55d686ed8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 --2- 172.21.15.191:0/4164840161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x55d6879c1800 tx=0x55d686ce1800 comp rx=0 tx=0).stop 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 -- 172.21.15.191:0/4164840161 shutdown_connections 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 --2- 172.21.15.191:0/4164840161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 -- 172.21.15.191:0/4164840161 >> 172.21.15.191:0/4164840161 conn(0x55d686bf0400 msgr2=0x55d686cdd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 -- 172.21.15.191:0/4164840161 shutdown_connections 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.764+0000 7f93bef19a80 1 -- 172.21.15.191:0/4164840161 wait complete. 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bef19a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bef19a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d686c923c0 con 0x55d686bf0800 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bcec3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bcec3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41148/0 (socket says 172.21.15.191:41148) 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bcec3700 1 -- 172.21.15.191:0/1023067853 learned_addr learned my addr 172.21.15.191:0/1023067853 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bcec3700 1 -- 172.21.15.191:0/1023067853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c19a00 con 0x55d686bf0800 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bcec3700 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x55d6879c1ec0 tx=0x55d686ce1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93b9ebd700 1 -- 172.21.15.191:0/1023067853 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca9600 con 0x55d686bf0800 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93b9ebd700 1 -- 172.21.15.191:0/1023067853 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d686f1bb00 con 0x55d686bf0800 2024-04-27T04:36:29.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d686c19860 con 0x55d686bf0800 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.765+0000 7f93b9ebd700 1 -- 172.21.15.191:0/1023067853 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca98c0 con 0x55d686bf0800 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d686c196c0 con 0x55d686bf0800 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93b9ebd700 1 -- 172.21.15.191:0/1023067853 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d687992b00 con 0x55d686bf0800 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93bef19a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93bef19a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d686c92000 con 0x55d686bf1400 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.766+0000 7f93b9ebd700 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82000 0x55d686ed9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93b9ebd700 1 -- 172.21.15.191:0/1023067853 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55d686c6b180 con 0x55d686bf0800 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bd6c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bd6c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41154/0 (socket says 172.21.15.191:41154) 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bd6c4700 1 -- 172.21.15.191:0/1422653727 learned_addr learned my addr 172.21.15.191:0/1422653727 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bcec3700 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82000 0x55d686ed9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bcec3700 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82000 0x55d686ed9080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55d687a71050 tx=0x55d686f48b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bd6c4700 1 -- 172.21.15.191:0/1422653727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c18000 con 0x55d686bf1400 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/1422653727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55d687a70900 tx=0x55d686f48e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d7ae5c389277cb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93b76b8700 1 -- 172.21.15.191:0/1422653727 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca9600 con 0x55d686bf1400 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93b76b8700 1 -- 172.21.15.191:0/1422653727 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d686c91c80 con 0x55d686bf1400 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.767+0000 7f93b76b8700 1 -- 172.21.15.191:0/1422653727 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca98c0 con 0x55d686bf1400 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- 172.21.15.191:0/1422653727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 msgr2=0x55d686ed8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1422653727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55d687a70900 tx=0x55d686f48e40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- 172.21.15.191:0/1422653727 shutdown_connections 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1422653727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- 172.21.15.191:0/1422653727 >> 172.21.15.191:0/1422653727 conn(0x55d686bf0c00 msgr2=0x55d686cdd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- 172.21.15.191:0/1422653727 shutdown_connections 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- 172.21.15.191:0/1422653727 wait complete. 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bef19a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d686c2c640 con 0x55d686bf1400 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bd6c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bd6c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41158/0 (socket says 172.21.15.191:41158) 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.768+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 learned_addr learned my addr 172.21.15.191:0/2160786291 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c18ea0 con 0x55d686bf1400 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55d687a706c0 tx=0x55d686ce1ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93b5eb5700 1 -- 172.21.15.191:0/2160786291 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca9760 con 0x55d686bf1400 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93b5eb5700 1 -- 172.21.15.191:0/2160786291 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d686c91b00 con 0x55d686bf1400 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93b5eb5700 1 -- 172.21.15.191:0/2160786291 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d686ca9600 con 0x55d686bf1400 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d686c18820 con 0x55d686bf1400 2024-04-27T04:36:29.777 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.769+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d686c189c0 con 0x55d686bf1400 2024-04-27T04:36:29.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93b5eb5700 1 -- 172.21.15.191:0/2160786291 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d687992dc0 con 0x55d686bf1400 2024-04-27T04:36:29.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93b5eb5700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82c00 0x55d686ed9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93b5eb5700 1 -- 172.21.15.191:0/2160786291 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55d686c6b340 con 0x55d686bf1400 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82c00 0x55d686ed9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d686bf1800 0x55d686ed9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686bf1c00 con 0x55d686bf1800 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d686bf1800 0x55d686ed9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82c00 0x55d686ed9600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55d6879c0030 tx=0x55d68702e0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.770+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d686bf1800 0x55d686ed9b80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 realm 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83000 con 0x55d686bf1800 2024-04-27T04:36:29.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83400 con 0x55d686bf1800 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83800 con 0x55d686bf1800 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d686f83c00 0x55d686eda100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.771+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83000 con 0x55d686f83c00 2024-04-27T04:36:29.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.772+0000 7f93bcec3700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d686f83c00 0x55d686eda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.772+0000 7f93bcec3700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d686f83c00 0x55d686eda100 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.772+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.772+0000 7f93bef19a80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.772+0000 7f93bef19a80 20 searching for the correct realm 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.772+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83400 con 0x55d686f83c00 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bcec3700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d686f83400 0x55d686edac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83000 con 0x55d686f83400 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d686f83400 0x55d686edac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d686f83400 0x55d686edac00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d686f83000 0x55d686edb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.773+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d686f83000 0x55d686edb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d686f83000 0x55d686edb180 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.774+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83400 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.775+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83400 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.776+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.777+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.778+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.778+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.778+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.778+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.778+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.778+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.798 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83400 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.779+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83400 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83400 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83c00 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.799 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.780+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83400 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686bf1c00 con 0x55d686bf1800 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.781+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042400 con 0x55d686bf1800 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042800 con 0x55d686f83c00 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 zone default found 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 4 Realm: () 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 zonegroup default 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.801 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042c00 con 0x55d686f83000 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043000 con 0x55d686f83c00 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.782+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043400 con 0x55d686f83c00 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.783+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.783+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.783+0000 7f93bef19a80 20 started sync module instance, tier type = 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.783+0000 7f93bef19a80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.783+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043800 con 0x55d686f83400 2024-04-27T04:36:29.802 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.784+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.784+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94379875040000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042800 con 0x55d686f83400 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.784+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043000 con 0x55d686bf1800 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.785+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94379875040000] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.785+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.785+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94379875041152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043400 con 0x55d686bf1800 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.785+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042400 con 0x55d686f83c00 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.786+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94379875041152] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.786+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.786+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94379875042304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042c00 con 0x55d686f83c00 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.786+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686bf1c00 con 0x55d686f83400 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.787+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.787+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94379875042304] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.787+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94379875043456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d686f83800 con 0x55d686f83400 2024-04-27T04:36:29.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.787+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686bf1800 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.788+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.788+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94379875044608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043800 con 0x55d686bf1800 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.788+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042800 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.788+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94379875043456] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.789+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94379875044608] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.789+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.789+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94379875045760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043c00 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.789+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043000 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.790+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94379875045760] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.791+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.791+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94379875046912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043400 con 0x55d686f83000 2024-04-27T04:36:29.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.791+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042000 con 0x55d686f83000 2024-04-27T04:36:29.815 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94379875046912] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(58 notify.7 [create] v33'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94379875048064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043800 con 0x55d686f83000 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=0 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=1 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=2 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=3 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=4 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=5 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.792+0000 7f93bef19a80 20 add_watcher() i=6 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.793+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94379875048064] v33'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.793+0000 7f93bef19a80 20 add_watcher() i=7 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.793+0000 7f93bef19a80 2 all 8 watchers are set, enabling cache 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.793+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042800 con 0x55d686bf1800 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.793+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.793+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94379876605952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043c00 con 0x55d686bf1800 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.794+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94379876605952] v33'243 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.794+0000 7f93a3e91700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.794+0000 7f93bef19a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.794+0000 7f93bef19a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bef19a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bef19a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d686c2c780 con 0x55d687043400 2024-04-27T04:36:29.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bc6c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bc6c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41160/0 (socket says 172.21.15.191:41160) 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2310582235 learned_addr learned my addr 172.21.15.191:0/2310582235 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2310582235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c18b60 con 0x55d687043400 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.795+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/2310582235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55d686fdb7d0 tx=0x55d686f49f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6dd137b5d03a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93a3690700 1 -- 172.21.15.191:0/2310582235 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d68705e000 con 0x55d687043400 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93a3690700 1 -- 172.21.15.191:0/2310582235 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d687056180 con 0x55d687043400 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93a3690700 1 -- 172.21.15.191:0/2310582235 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d68705e160 con 0x55d687043400 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 -- 172.21.15.191:0/2310582235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 msgr2=0x55d686eda680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2310582235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55d686fdb7d0 tx=0x55d686f49f80 comp rx=0 tx=0).stop 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 -- 172.21.15.191:0/2310582235 shutdown_connections 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2310582235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 -- 172.21.15.191:0/2310582235 >> 172.21.15.191:0/2310582235 conn(0x55d687043000 msgr2=0x55d686cde000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 -- 172.21.15.191:0/2310582235 shutdown_connections 2024-04-27T04:36:29.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.796+0000 7f93bef19a80 1 -- 172.21.15.191:0/2310582235 wait complete. 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bef19a80 1 Processor -- start 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bef19a80 1 -- start start 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bef19a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bef19a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d686c92000 con 0x55d687043400 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bc6c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bc6c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41176/0 (socket says 172.21.15.191:41176) 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bc6c2700 1 -- 172.21.15.191:0/1768745788 learned_addr learned my addr 172.21.15.191:0/1768745788 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bc6c2700 1 -- 172.21.15.191:0/1768745788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d686c18d00 con 0x55d687043400 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bc6c2700 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55d687061200 tx=0x55d68702e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93a1e8d700 1 -- 172.21.15.191:0/1768745788 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d68705e2c0 con 0x55d687043400 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93a1e8d700 1 -- 172.21.15.191:0/1768745788 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d687056300 con 0x55d687043400 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.797+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d686c18340 con 0x55d687043400 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.798+0000 7f93a1e8d700 1 -- 172.21.15.191:0/1768745788 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d68705e000 con 0x55d687043400 2024-04-27T04:36:29.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.798+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d686c18b60 con 0x55d687043400 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.798+0000 7f93a1e8d700 1 -- 172.21.15.191:0/1768745788 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d687993b80 con 0x55d687043400 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.798+0000 7f93a1e8d700 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d6870e4000 0x55d686edb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.798+0000 7f93a1e8d700 1 -- 172.21.15.191:0/1768745788 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55d686c6b500 con 0x55d687043400 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.799+0000 7f93bef19a80 5 note: GC not initialized 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.799+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d6870e4000 0x55d686edb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.799+0000 7f93bd6c4700 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d6870e4000 0x55d686edb700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d686fdb470 tx=0x55d68702e360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.799+0000 7f939f688700 20 reqs_thread_entry: start 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.799+0000 7f93bef19a80 20 init_complete bucket index max shards: 11 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f93bef19a80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f939d684700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:36:59 2024 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f939ce62700 20 reqs_thread_entry: start 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f93bef19a80 20 rados->read ofs=0 len=0 2024-04-27T04:36:29.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687043800 con 0x55d686bf1800 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.800+0000 7f93bef19a80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:could not fetch user info: no user info saved 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.801+0000 7f93a3e91700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.801+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94379876605952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d687042800 con 0x55d686bf1800 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.802+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94379876605952] v33'244 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.802+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94379875040000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e4400 con 0x55d686f83400 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.803+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94379875040000] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.803+0000 7f93bef19a80 20 remove_watcher() i=0 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.803+0000 7f93bef19a80 2 removed watcher, disabling cache 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.803+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94379875041152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e4800 con 0x55d686bf1800 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.804+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94379875041152] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.804+0000 7f93bef19a80 20 remove_watcher() i=1 2024-04-27T04:36:29.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.804+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94379875042304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e4c00 con 0x55d686f83c00 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.805+0000 7f93bcec3700 1 -- 172.21.15.191:0/2160786291 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94379875042304] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035200 con 0x55d686f83c00 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.805+0000 7f93bef19a80 20 remove_watcher() i=2 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.805+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94379875043456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e5000 con 0x55d686f83400 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.806+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94379875043456] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687035440 con 0x55d686f83400 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.806+0000 7f93bef19a80 20 remove_watcher() i=3 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.806+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94379875044608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e5400 con 0x55d686bf1800 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.807+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94379875044608] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686bf1800 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.807+0000 7f93bef19a80 20 remove_watcher() i=4 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.807+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94379875045760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e5800 con 0x55d686f83000 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.808+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94379875045760] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.808+0000 7f93bef19a80 20 remove_watcher() i=5 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.808+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94379875046912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e5c00 con 0x55d686f83000 2024-04-27T04:36:29.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.809+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94379875046912] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.809+0000 7f93bef19a80 20 remove_watcher() i=6 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.809+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94379875048064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d6870e4800 con 0x55d686f83000 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bd6c4700 1 -- 172.21.15.191:0/2160786291 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94379875048064] v33'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d687034b40 con 0x55d686f83000 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 20 remove_watcher() i=7 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d686f83000 msgr2=0x55d686edb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55d686f83000 0x55d686edb180 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d686bf1800 msgr2=0x55d686ed9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55d686bf1800 0x55d686ed9b80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d686f83400 msgr2=0x55d686edac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55d686f83400 0x55d686edac00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d686f83c00 msgr2=0x55d686eda100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55d686f83c00 0x55d686eda100 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82c00 msgr2=0x55d686ed9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82c00 0x55d686ed9600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55d6879c0030 tx=0x55d68702e0c0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 msgr2=0x55d686ed8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 --2- 172.21.15.191:0/2160786291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf1400 0x55d686ed8b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55d687a706c0 tx=0x55d686ce1ce0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 reap_dead start 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bc6c2700 1 -- 172.21.15.191:0/2160786291 reap_dead start 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 shutdown_connections 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 >> 172.21.15.191:0/2160786291 conn(0x55d686bf0c00 msgr2=0x55d686cdd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 shutdown_connections 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.810+0000 7f93bef19a80 1 -- 172.21.15.191:0/2160786291 wait complete. 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d6870e4000 msgr2=0x55d686edb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d6870e4000 0x55d686edb700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d686fdb470 tx=0x55d68702e360 comp rx=0 tx=0).stop 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 msgr2=0x55d686eda680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55d687061200 tx=0x55d68702e4e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 shutdown_connections 2024-04-27T04:36:29.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d6870e4000 0x55d686edb700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1768745788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d687043400 0x55d686eda680 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 >> 172.21.15.191:0/1768745788 conn(0x55d687043000 msgr2=0x55d686cde000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 shutdown_connections 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1768745788 wait complete. 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82000 msgr2=0x55d686ed9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82000 0x55d686ed9080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55d687a71050 tx=0x55d686f48b40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 msgr2=0x55d686ed8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x55d6879c1ec0 tx=0x55d686ce1980 comp rx=0 tx=0).stop 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 shutdown_connections 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55d686f82000 0x55d686ed9080 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 --2- 172.21.15.191:0/1023067853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d686bf0800 0x55d686ed8580 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 >> 172.21.15.191:0/1023067853 conn(0x55d686bf0400 msgr2=0x55d686cdd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 shutdown_connections 2024-04-27T04:36:29.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.811+0000 7f93bef19a80 1 -- 172.21.15.191:0/1023067853 wait complete. 2024-04-27T04:36:29.826 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-04-27T04:36:29.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-04-27T04:36:29.826 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-04-27T04:36:29.870 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:29.870 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f7882332a80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556a75ce8c00 0x556a75fd0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f7882332a80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556a76013500 con 0x556a75ce8800 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f7882332a80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x556a75d8aa00 con 0x556a75ce8c00 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f787fadb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556a75ce8c00 0x556a75fd0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f787fadb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x556a75ce8c00 0x556a75fd0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41184/0 (socket says 172.21.15.191:41184) 2024-04-27T04:36:29.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.869+0000 7f787fadb700 1 -- 172.21.15.191:0/4104764348 learned_addr learned my addr 172.21.15.191:0/4104764348 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.884 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787fadb700 1 -- 172.21.15.191:0/4104764348 >> v1:172.21.15.138:6789/0 conn(0x556a75ce8800 legacy=0x556a75dd4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787fadb700 1 -- 172.21.15.191:0/4104764348 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d11d40 con 0x556a75ce8c00 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787f2da700 1 -- 172.21.15.191:0/4104764348 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900831308 0 0) 0x556a76ab6000 con 0x556a75ce8800 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787fadb700 1 --2- 172.21.15.191:0/4104764348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8c00 0x556a75fd0580 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x556a75fcb890 tx=0x556a75dd9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16150ddf282fb969 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787f2da700 1 -- 172.21.15.191:0/4104764348 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da14a0 con 0x556a75ce8c00 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787f2da700 1 -- 172.21.15.191:0/4104764348 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a76013800 con 0x556a75ce8c00 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.870+0000 7f787f2da700 1 -- 172.21.15.191:0/4104764348 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1600 con 0x556a75ce8c00 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- 172.21.15.191:0/4104764348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8c00 msgr2=0x556a75fd0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 --2- 172.21.15.191:0/4104764348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8c00 0x556a75fd0580 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x556a75fcb890 tx=0x556a75dd9920 comp rx=0 tx=0).stop 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- 172.21.15.191:0/4104764348 shutdown_connections 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 --2- 172.21.15.191:0/4104764348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8c00 0x556a75fd0580 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- 172.21.15.191:0/4104764348 >> 172.21.15.191:0/4104764348 conn(0x556a75ce8400 msgr2=0x556a75dd4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- 172.21.15.191:0/4104764348 shutdown_connections 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- 172.21.15.191:0/4104764348 wait complete. 2024-04-27T04:36:29.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.871+0000 7f7882332a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a75d8a500 con 0x556a75ce8800 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41196/0 (socket says 172.21.15.191:41196) 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f78802dc700 1 -- 172.21.15.191:0/3088960952 learned_addr learned my addr 172.21.15.191:0/3088960952 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f78802dc700 1 -- 172.21.15.191:0/3088960952 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d11ba0 con 0x556a75ce8800 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f78802dc700 1 --2- 172.21.15.191:0/3088960952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x556a76ab3830 tx=0x556a75dd9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0da0cceb3b1850 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f787ead9700 1 -- 172.21.15.191:0/3088960952 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da14a0 con 0x556a75ce8800 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f787ead9700 1 -- 172.21.15.191:0/3088960952 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a76013b00 con 0x556a75ce8800 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f787ead9700 1 -- 172.21.15.191:0/3088960952 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1600 con 0x556a75ce8800 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f7882332a80 1 -- 172.21.15.191:0/3088960952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 msgr2=0x556a75fd0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.872+0000 7f7882332a80 1 --2- 172.21.15.191:0/3088960952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x556a76ab3830 tx=0x556a75dd9b60 comp rx=0 tx=0).stop 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 -- 172.21.15.191:0/3088960952 shutdown_connections 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 --2- 172.21.15.191:0/3088960952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 -- 172.21.15.191:0/3088960952 >> 172.21.15.191:0/3088960952 conn(0x556a75ce8400 msgr2=0x556a75dd5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 -- 172.21.15.191:0/3088960952 shutdown_connections 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 -- 172.21.15.191:0/3088960952 wait complete. 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f7882332a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a75d8a3c0 con 0x556a75ce8800 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41212/0 (socket says 172.21.15.191:41212) 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f78802dc700 1 -- 172.21.15.191:0/2254335435 learned_addr learned my addr 172.21.15.191:0/2254335435 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.873+0000 7f78802dc700 1 -- 172.21.15.191:0/2254335435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d11a00 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f78802dc700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x556a76ab3a70 tx=0x556a75dd9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f787d2d6700 1 -- 172.21.15.191:0/2254335435 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da18c0 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f787d2d6700 1 -- 172.21.15.191:0/2254335435 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a75d89c80 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556a75d11860 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f787d2d6700 1 -- 172.21.15.191:0/2254335435 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da14a0 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a75d116c0 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f787d2d6700 1 -- 172.21.15.191:0/2254335435 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556a76a8ab00 con 0x556a75ce8800 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.874+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7882332a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7882332a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a75d8a000 con 0x556a75ce9400 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f787d2d6700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607c000 0x556a75fd1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.888 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f787d2d6700 1 -- 172.21.15.191:0/2254335435 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556a75d63180 con 0x556a75ce8800 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f78802dc700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607c000 0x556a75fd1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7880add700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7880add700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41220/0 (socket says 172.21.15.191:41220) 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7880add700 1 -- 172.21.15.191:0/1962375315 learned_addr learned my addr 172.21.15.191:0/1962375315 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f78802dc700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607c000 0x556a75fd1080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556a76b66fc0 tx=0x556a76036240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7880add700 1 -- 172.21.15.191:0/1962375315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d10000 con 0x556a75ce9400 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f7880add700 1 --2- 172.21.15.191:0/1962375315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x556a76b66600 tx=0x556a760370e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85de73b187839aab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f787aad1700 1 -- 172.21.15.191:0/1962375315 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da18c0 con 0x556a75ce9400 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.875+0000 7f787aad1700 1 -- 172.21.15.191:0/1962375315 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a76012300 con 0x556a75ce9400 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f787aad1700 1 -- 172.21.15.191:0/1962375315 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da14a0 con 0x556a75ce9400 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- 172.21.15.191:0/1962375315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 msgr2=0x556a75fd0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 --2- 172.21.15.191:0/1962375315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x556a76b66600 tx=0x556a760370e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- 172.21.15.191:0/1962375315 shutdown_connections 2024-04-27T04:36:29.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 --2- 172.21.15.191:0/1962375315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- 172.21.15.191:0/1962375315 >> 172.21.15.191:0/1962375315 conn(0x556a75ce8c00 msgr2=0x556a75dd5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- 172.21.15.191:0/1962375315 shutdown_connections 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- 172.21.15.191:0/1962375315 wait complete. 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.876+0000 7f7882332a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a75d24640 con 0x556a75ce9400 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7880add700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7880add700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41230/0 (socket says 172.21.15.191:41230) 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7880add700 1 -- 172.21.15.191:0/3488636339 learned_addr learned my addr 172.21.15.191:0/3488636339 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7880add700 1 -- 172.21.15.191:0/3488636339 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d10ea0 con 0x556a75ce9400 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7880add700 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x556a76b67f50 tx=0x556a76036fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f78792ce700 1 -- 172.21.15.191:0/3488636339 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1b80 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f78792ce700 1 -- 172.21.15.191:0/3488636339 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a75d89c80 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556a75d10820 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f78792ce700 1 -- 172.21.15.191:0/3488636339 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da18c0 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.877+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a75d109c0 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f78792ce700 1 -- 172.21.15.191:0/3488636339 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556a76a8adc0 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7882332a80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7882332a80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f78792ce700 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607cc00 0x556a75fd1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f78792ce700 1 -- 172.21.15.191:0/3488636339 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556a75d63340 con 0x556a75ce9400 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7880add700 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607cc00 0x556a75fd1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7882332a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.878+0000 7f7882332a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a75d248c0 con 0x556a75ce9c00 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7880add700 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607cc00 0x556a75fd1600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x556a760d0540 tx=0x556a760377a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41246/0 (socket says 172.21.15.191:41246) 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78802dc700 1 -- 172.21.15.191:0/1994958350 learned_addr learned my addr 172.21.15.191:0/1994958350 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78802dc700 1 -- 172.21.15.191:0/1994958350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d10b60 con 0x556a75ce9c00 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78802dc700 1 --2- 172.21.15.191:0/1994958350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x556a76b66270 tx=0x556a76036ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f5ccb9c8d0b80c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78672aa700 1 -- 172.21.15.191:0/1994958350 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1600 con 0x556a75ce9c00 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78672aa700 1 -- 172.21.15.191:0/1994958350 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a75d89c80 con 0x556a75ce9c00 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f78672aa700 1 -- 172.21.15.191:0/1994958350 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1a20 con 0x556a75ce9c00 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 -- 172.21.15.191:0/1994958350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 msgr2=0x556a75fd1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 --2- 172.21.15.191:0/1994958350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x556a76b66270 tx=0x556a76036ea0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 -- 172.21.15.191:0/1994958350 shutdown_connections 2024-04-27T04:36:29.892 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 --2- 172.21.15.191:0/1994958350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 -- 172.21.15.191:0/1994958350 >> 172.21.15.191:0/1994958350 conn(0x556a75ce9800 msgr2=0x556a75dd6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 -- 172.21.15.191:0/1994958350 shutdown_connections 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.879+0000 7f7882332a80 1 -- 172.21.15.191:0/1994958350 wait complete. 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f7882332a80 1 Processor -- start 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f7882332a80 1 -- start start 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f7882332a80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f7882332a80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a75d24f00 con 0x556a75ce9c00 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f78802dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41250/0 (socket says 172.21.15.191:41250) 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f78802dc700 1 -- 172.21.15.191:0/2805067247 learned_addr learned my addr 172.21.15.191:0/2805067247 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f78802dc700 1 -- 172.21.15.191:0/2805067247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a75d10d00 con 0x556a75ce9c00 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f78802dc700 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x556a76138060 tx=0x556a760361e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.880+0000 7f7865aa7700 1 -- 172.21.15.191:0/2805067247 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1600 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7865aa7700 1 -- 172.21.15.191:0/2805067247 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a7613a000 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556a75d10340 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7865aa7700 1 -- 172.21.15.191:0/2805067247 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556a75da1a20 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a7613c000 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7865aa7700 1 -- 172.21.15.191:0/2805067247 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556a76a8bb80 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7865aa7700 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7619c000 0x556a75fd2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7865aa7700 1 -- 172.21.15.191:0/2805067247 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x556a75d63500 con 0x556a75ce9c00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f787fadb700 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7619c000 0x556a75fd2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.881+0000 7f7863aa3700 20 reqs_thread_entry: start 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556a7607dc00 0x556a75fd2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556a7619c400 con 0x556a7607dc00 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7880add700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556a7607dc00 0x556a75fd2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f787fadb700 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7619c000 0x556a75fd2c00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x556a75fcb890 tx=0x556a75dd9800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7880add700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556a7607dc00 0x556a75fd2100 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7880add700 1 -- 172.21.15.191:0/2254335435 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556a761b0240 con 0x556a7607dc00 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556a7619c800 0x556a75fd2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:2 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556a7619cc00 con 0x556a7619c800 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.882+0000 7f78802dc700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556a7619c800 0x556a75fd2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.883+0000 7f78802dc700 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556a7619c800 0x556a75fd2680 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.883+0000 7f78802dc700 1 -- 172.21.15.191:0/2254335435 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(2 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x556a761b0480 con 0x556a7619c800 2024-04-27T04:36:29.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.883+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607cc00 msgr2=0x556a75fd1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.883+0000 7f7882332a80 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607cc00 0x556a75fd1600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x556a760d0540 tx=0x556a760377a0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.883+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 msgr2=0x556a75fd0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.883+0000 7f7882332a80 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x556a76b67f50 tx=0x556a76036fc0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 shutdown_connections 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607cc00 0x556a75fd1600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/3488636339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9400 0x556a75fd0b00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 >> 172.21.15.191:0/3488636339 conn(0x556a75ce8c00 msgr2=0x556a75dd5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 shutdown_connections 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/3488636339 wait complete. 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7619c000 msgr2=0x556a75fd2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7619c000 0x556a75fd2c00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x556a75fcb890 tx=0x556a75dd9800 comp rx=0 tx=0).stop 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 msgr2=0x556a75fd1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x556a76138060 tx=0x556a760361e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.898 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 shutdown_connections 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7619c000 0x556a75fd2c00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2805067247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce9c00 0x556a75fd1b80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 >> 172.21.15.191:0/2805067247 conn(0x556a75ce9800 msgr2=0x556a75dd6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 shutdown_connections 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2805067247 wait complete. 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556a7607dc00 msgr2=0x556a75fd2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556a7607dc00 0x556a75fd2100 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556a7619c800 msgr2=0x556a75fd2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556a7619c800 0x556a75fd2680 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607c000 msgr2=0x556a75fd1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607c000 0x556a75fd1080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556a76b66fc0 tx=0x556a76036240 comp rx=0 tx=0).stop 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 msgr2=0x556a75fd0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x556a76ab3a70 tx=0x556a75dd9a40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 shutdown_connections 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x556a7619c800 0x556a75fd2680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x556a7607dc00 0x556a75fd2100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a75ce8800 0x556a75fd0580 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 --2- 172.21.15.191:0/2254335435 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x556a7607c000 0x556a75fd1080 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 >> 172.21.15.191:0/2254335435 conn(0x556a75ce8400 msgr2=0x556a75dd5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 shutdown_connections 2024-04-27T04:36:29.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.884+0000 7f7882332a80 1 -- 172.21.15.191:0/2254335435 wait complete. 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "control_pool": "default.rgw.control", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "log_pool": "default.rgw.log", 2024-04-27T04:36:29.901 INFO:teuthology.orchestra.run.smithi191.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "system_key": { 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "", 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "" 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:29.902 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_pools": [ 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "default-placement", 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": true 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.903 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.904 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:29.904 INFO:teuthology.orchestra.run.smithi191.stdout: "realm_id": "", 2024-04-27T04:36:29.904 INFO:teuthology.orchestra.run.smithi191.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T04:36:29.904 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:29.904 DEBUG:tasks.util.rgw: json result: {'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T04:36:29.904 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-04-27T04:36:29.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-04-27T04:36:29.904 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-04-27T04:36:29.948 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:29.948 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed5591da80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565167cd2c00 0x565167fba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed5591da80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x565167ffd500 con 0x565167cd2800 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed5591da80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x565167d74a00 con 0x565167cd2c00 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed530c6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x565167cd2c00 0x565167fba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed538c7700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x565167cd2800 0x565167dbe800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46360/0 (socket says 172.21.15.191:46360) 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.947+0000 7fed538c7700 1 -- 172.21.15.191:0/1410034330 learned_addr learned my addr 172.21.15.191:0/1410034330 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:29.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed530c6700 1 -- 172.21.15.191:0/1410034330 >> v1:172.21.15.138:6789/0 conn(0x565167cd2800 legacy=0x565167dbe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed530c6700 1 -- 172.21.15.191:0/1410034330 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfbd40 con 0x565167cd2c00 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed528c5700 1 -- 172.21.15.191:0/1410034330 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722535981 0 0) 0x565168aa0000 con 0x565167cd2800 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed530c6700 1 --2- 172.21.15.191:0/1410034330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2c00 0x565167fba580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x565167fb5890 tx=0x565167dc3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b601d4278e5b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed528c5700 1 -- 172.21.15.191:0/1410034330 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b4a0 con 0x565167cd2c00 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed528c5700 1 -- 172.21.15.191:0/1410034330 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565167ffd800 con 0x565167cd2c00 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed528c5700 1 -- 172.21.15.191:0/1410034330 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b600 con 0x565167cd2c00 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 -- 172.21.15.191:0/1410034330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2c00 msgr2=0x565167fba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 --2- 172.21.15.191:0/1410034330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2c00 0x565167fba580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x565167fb5890 tx=0x565167dc3920 comp rx=0 tx=0).stop 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 -- 172.21.15.191:0/1410034330 shutdown_connections 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 --2- 172.21.15.191:0/1410034330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2c00 0x565167fba580 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 -- 172.21.15.191:0/1410034330 >> 172.21.15.191:0/1410034330 conn(0x565167cd2400 msgr2=0x565167dbe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 -- 172.21.15.191:0/1410034330 shutdown_connections 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.948+0000 7fed5591da80 1 -- 172.21.15.191:0/1410034330 wait complete. 2024-04-27T04:36:29.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed5591da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed5591da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167d74500 con 0x565167cd2800 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41268/0 (socket says 172.21.15.191:41268) 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.949+0000 7fed538c7700 1 -- 172.21.15.191:0/3060722352 learned_addr learned my addr 172.21.15.191:0/3060722352 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed538c7700 1 -- 172.21.15.191:0/3060722352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfbba0 con 0x565167cd2800 2024-04-27T04:36:29.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed538c7700 1 --2- 172.21.15.191:0/3060722352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x565168aa3770 tx=0x565167dc3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3097b8efe09d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed520c4700 1 -- 172.21.15.191:0/3060722352 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b1e0 con 0x565167cd2800 2024-04-27T04:36:29.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed520c4700 1 -- 172.21.15.191:0/3060722352 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565167ffdb00 con 0x565167cd2800 2024-04-27T04:36:29.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed520c4700 1 -- 172.21.15.191:0/3060722352 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b4a0 con 0x565167cd2800 2024-04-27T04:36:29.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 -- 172.21.15.191:0/3060722352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 msgr2=0x565167fba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 --2- 172.21.15.191:0/3060722352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x565168aa3770 tx=0x565167dc3b60 comp rx=0 tx=0).stop 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 -- 172.21.15.191:0/3060722352 shutdown_connections 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 --2- 172.21.15.191:0/3060722352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 -- 172.21.15.191:0/3060722352 >> 172.21.15.191:0/3060722352 conn(0x565167cd2400 msgr2=0x565167dbf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 -- 172.21.15.191:0/3060722352 shutdown_connections 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.950+0000 7fed5591da80 1 -- 172.21.15.191:0/3060722352 wait complete. 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed5591da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed5591da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167d743c0 con 0x565167cd2800 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41270/0 (socket says 172.21.15.191:41270) 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 learned_addr learned my addr 172.21.15.191:0/215343998 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfba00 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.951+0000 7fed538c7700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x565168aa3e30 tx=0x565167dc3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed508c1700 1 -- 172.21.15.191:0/215343998 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b600 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed508c1700 1 -- 172.21.15.191:0/215343998 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565167d73c80 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565167cfb860 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed508c1700 1 -- 172.21.15.191:0/215343998 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b1e0 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565167cfb6c0 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed508c1700 1 -- 172.21.15.191:0/215343998 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565168a74b00 con 0x565167cd2800 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed5591da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed5591da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167d74000 con 0x565167cd3400 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed508c1700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062000 0x565167fbb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed508c1700 1 -- 172.21.15.191:0/215343998 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x565167d4d180 con 0x565167cd2800 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.952+0000 7fed540c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed540c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41280/0 (socket says 172.21.15.191:41280) 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed540c8700 1 -- 172.21.15.191:0/3082720012 learned_addr learned my addr 172.21.15.191:0/3082720012 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed538c7700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062000 0x565167fbb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed540c8700 1 -- 172.21.15.191:0/3082720012 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfa000 con 0x565167cd3400 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed538c7700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062000 0x565167fbb080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x565168b64f00 tx=0x56516801e240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed540c8700 1 --2- 172.21.15.191:0/3082720012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x565168b643c0 tx=0x56516801f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abcb1f5a99d64b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed4e0bc700 1 -- 172.21.15.191:0/3082720012 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b600 con 0x565167cd3400 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed4e0bc700 1 -- 172.21.15.191:0/3082720012 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565167ffc300 con 0x565167cd3400 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed4e0bc700 1 -- 172.21.15.191:0/3082720012 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b1e0 con 0x565167cd3400 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 -- 172.21.15.191:0/3082720012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 msgr2=0x565167fbab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 --2- 172.21.15.191:0/3082720012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x565168b643c0 tx=0x56516801f0e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 -- 172.21.15.191:0/3082720012 shutdown_connections 2024-04-27T04:36:29.971 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 --2- 172.21.15.191:0/3082720012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 -- 172.21.15.191:0/3082720012 >> 172.21.15.191:0/3082720012 conn(0x565167cd2c00 msgr2=0x565167dbf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 -- 172.21.15.191:0/3082720012 shutdown_connections 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.953+0000 7fed5591da80 1 -- 172.21.15.191:0/3082720012 wait complete. 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed5591da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed5591da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167d0e640 con 0x565167cd3400 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed540c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed540c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41292/0 (socket says 172.21.15.191:41292) 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed540c8700 1 -- 172.21.15.191:0/844231778 learned_addr learned my addr 172.21.15.191:0/844231778 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.954+0000 7fed540c8700 1 -- 172.21.15.191:0/844231778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfaea0 con 0x565167cd3400 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.955+0000 7fed540c8700 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x565168b64420 tx=0x56516801ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.955+0000 7fed4c8b9700 1 -- 172.21.15.191:0/844231778 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8ba20 con 0x565167cd3400 2024-04-27T04:36:29.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.955+0000 7fed4c8b9700 1 -- 172.21.15.191:0/844231778 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565167d73c80 con 0x565167cd3400 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.955+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565167cfa820 con 0x565167cd3400 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.955+0000 7fed4c8b9700 1 -- 172.21.15.191:0/844231778 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b600 con 0x565167cd3400 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.955+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565167cfa9c0 con 0x565167cd3400 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed4c8b9700 1 -- 172.21.15.191:0/844231778 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565168a74dc0 con 0x565167cd3400 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed5591da80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed5591da80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed4c8b9700 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062c00 0x565167fbb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed4c8b9700 1 -- 172.21.15.191:0/844231778 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x565167d4d340 con 0x565167cd3400 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed540c8700 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062c00 0x565167fbb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed5591da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed5591da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167d0e8c0 con 0x565167cd3c00 2024-04-27T04:36:29.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed540c8700 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062c00 0x565167fbb600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5651680c88a0 tx=0x56516801f7a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41304/0 (socket says 172.21.15.191:41304) 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.956+0000 7fed538c7700 1 -- 172.21.15.191:0/2291198715 learned_addr learned my addr 172.21.15.191:0/2291198715 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed538c7700 1 -- 172.21.15.191:0/2291198715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfab60 con 0x565167cd3c00 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed538c7700 1 --2- 172.21.15.191:0/2291198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x565168b64240 tx=0x56516801efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39ee88bbcf2a880 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed3a895700 1 -- 172.21.15.191:0/2291198715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b4a0 con 0x565167cd3c00 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed3a895700 1 -- 172.21.15.191:0/2291198715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565167d73c80 con 0x565167cd3c00 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed3a895700 1 -- 172.21.15.191:0/2291198715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b8c0 con 0x565167cd3c00 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 -- 172.21.15.191:0/2291198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 msgr2=0x565167fbbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 --2- 172.21.15.191:0/2291198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x565168b64240 tx=0x56516801efc0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 -- 172.21.15.191:0/2291198715 shutdown_connections 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 --2- 172.21.15.191:0/2291198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 -- 172.21.15.191:0/2291198715 >> 172.21.15.191:0/2291198715 conn(0x565167cd3800 msgr2=0x565167dc0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 -- 172.21.15.191:0/2291198715 shutdown_connections 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.957+0000 7fed5591da80 1 -- 172.21.15.191:0/2291198715 wait complete. 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed5591da80 1 Processor -- start 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed5591da80 1 -- start start 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed5591da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed5591da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167d0ef00 con 0x565167cd3c00 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed538c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41320/0 (socket says 172.21.15.191:41320) 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed538c7700 1 -- 172.21.15.191:0/4281234447 learned_addr learned my addr 172.21.15.191:0/4281234447 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed538c7700 1 -- 172.21.15.191:0/4281234447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167cfad00 con 0x565167cd3c00 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed538c7700 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x565168069800 tx=0x56516801e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed39092700 1 -- 172.21.15.191:0/4281234447 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b4a0 con 0x565167cd3c00 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed39092700 1 -- 172.21.15.191:0/4281234447 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565168128000 con 0x565167cd3c00 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565167cfa340 con 0x565167cd3c00 2024-04-27T04:36:29.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.958+0000 7fed39092700 1 -- 172.21.15.191:0/4281234447 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565167d8b8c0 con 0x565167cd3c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565168130000 con 0x565167cd3c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed39092700 1 -- 172.21.15.191:0/4281234447 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x565168a75b80 con 0x565167cd3c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed39092700 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56516818e000 0x565167fbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed39092700 1 -- 172.21.15.191:0/4281234447 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x565167d4d500 con 0x565167cd3c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed530c6700 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56516818e000 0x565167fbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed3708e700 20 reqs_thread_entry: start 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565168063c00 0x565167fbc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.959+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818e400 con 0x565168063c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.960+0000 7fed540c8700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565168063c00 0x565167fbc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.960+0000 7fed530c6700 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56516818e000 0x565167fbcc00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x565167fb5890 tx=0x565167dc3800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.960+0000 7fed540c8700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565168063c00 0x565167fbc100 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.960+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.960+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818e800 con 0x565168063c00 2024-04-27T04:36:29.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.960+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56516818ec00 0x565167fbc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818f000 con 0x56516818ec00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed538c7700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56516818ec00 0x565167fbc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed538c7700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56516818ec00 0x565167fbc680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818f400 con 0x565168063c00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.961+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:NOTICE: set zone's realm_id= 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818e400 con 0x56516818ec00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(5 [pgnls start_epoch 33 out=107b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed538c7700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56516818e400 0x565167fbd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818f000 con 0x56516818e400 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed530c6700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56516818e400 0x565167fbd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed530c6700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56516818e400 0x565167fbd180 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed530c6700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56516818f000 0x565167fbd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.962+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.963+0000 7fed540c8700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56516818f000 0x565167fbd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.963+0000 7fed540c8700 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56516818f000 0x565167fbd700 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.963+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.963+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818e400 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.964+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.965+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818e400 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.979 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.966+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(21 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.967+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.986 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818e400 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.968+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818e400 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818e400 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818ec00 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.987 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.969+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818f000 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2240 con 0x56516818f000 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x565168063c00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(35 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be000 con 0x56516818e400 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed530c6700 1 -- 172.21.15.191:0/215343998 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(36 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5651681a2b40 con 0x56516818e400 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818e800 con 0x565168063c00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.970+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.971+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:38 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818f400 con 0x56516818ec00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.971+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(38 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.971+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:39 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [create,call version.set in=58b,writefull 0~1036 in=1036b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818f800 con 0x56516818ec00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.972+0000 7fed538c7700 1 -- 172.21.15.191:0/215343998 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(39 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [create,call,writefull 0~1036] v33'3 uv3 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x5651681a2900 con 0x56516818ec00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.972+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56516818fc00 con 0x565168063c00 2024-04-27T04:36:29.988 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.973+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v33'3 uv3 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.973+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5651681be400 con 0x565168063c00 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.974+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v33'1 uv1 ondisk = 0) v8 ==== 199+0+0 (crc 0 0 0) 0x5651681a2240 con 0x565168063c00 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062c00 msgr2=0x565167fbb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062c00 0x565167fbb600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5651680c88a0 tx=0x56516801f7a0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 msgr2=0x565167fbab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x565168b64420 tx=0x56516801ed20 comp rx=0 tx=0).stop 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 shutdown_connections 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062c00 0x565167fbb600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/844231778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3400 0x565167fbab00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 >> 172.21.15.191:0/844231778 conn(0x565167cd2c00 msgr2=0x565167dbf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 shutdown_connections 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/844231778 wait complete. 2024-04-27T04:36:29.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56516818e000 msgr2=0x565167fbcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56516818e000 0x565167fbcc00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x565167fb5890 tx=0x565167dc3800 comp rx=0 tx=0).stop 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 msgr2=0x565167fbbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x565168069800 tx=0x56516801e1e0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 shutdown_connections 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x56516818e000 0x565167fbcc00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 --2- 172.21.15.191:0/4281234447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd3c00 0x565167fbbb80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 >> 172.21.15.191:0/4281234447 conn(0x565167cd3800 msgr2=0x565167dc0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 shutdown_connections 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.975+0000 7fed5591da80 1 -- 172.21.15.191:0/4281234447 wait complete. 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56516818f000 msgr2=0x565167fbd700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x56516818f000 0x565167fbd700 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565168063c00 msgr2=0x565167fbc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x565168063c00 0x565167fbc100 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.990 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56516818e400 msgr2=0x565167fbd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x56516818e400 0x565167fbd180 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56516818ec00 msgr2=0x565167fbc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x56516818ec00 0x565167fbc680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062000 msgr2=0x565167fbb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x565168062000 0x565167fbb080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x565168b64f00 tx=0x56516801e240 comp rx=0 tx=0).stop 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed540c8700 1 -- 172.21.15.191:0/215343998 reap_dead start 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 msgr2=0x565167fba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x565168aa3e30 tx=0x565167dc3a40 comp rx=0 tx=0).stop 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 shutdown_connections 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 --2- 172.21.15.191:0/215343998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565167cd2800 0x565167fba580 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 >> 172.21.15.191:0/215343998 conn(0x565167cd2400 msgr2=0x565167dbf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 shutdown_connections 2024-04-27T04:36:29.991 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:29.976+0000 7fed5591da80 1 -- 172.21.15.191:0/215343998 wait complete. 2024-04-27T04:36:29.992 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:29.992 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:36:29.992 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:36:29.992 INFO:teuthology.orchestra.run.smithi191.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T04:36:29.992 INFO:teuthology.orchestra.run.smithi191.stdout: "control_pool": "default.rgw.control", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "log_pool": "default.rgw.log", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "system_key": { 2024-04-27T04:36:29.993 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "", 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "" 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_pools": [ 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "default-placement", 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:36:29.994 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": true 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "new-placement", 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": ".rgw.buckets.index.2", 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": ".rgw.buckets.2" 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:36:29.995 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": false 2024-04-27T04:36:29.996 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.996 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:29.996 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:29.996 INFO:teuthology.orchestra.run.smithi191.stdout: "realm_id": "", 2024-04-27T04:36:29.996 INFO:teuthology.orchestra.run.smithi191.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T04:36:29.996 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:29.996 DEBUG:tasks.util.rgw: json result: {'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T04:36:29.996 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-04-27T04:36:29.996 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-04-27T04:36:29.997 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-04-27T04:36:30.076 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:30.076 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:30.078 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a90b3aa80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b43aa40c00 0x55b43ad28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a90b3aa80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b43ad6b500 con 0x55b43aa40800 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a90b3aa80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b43aae2a00 con 0x55b43aa40c00 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a8e2e3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55b43aa40c00 0x55b43ad28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a8eae4700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55b43aa40800 0x55b43ab2c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46372/0 (socket says 172.21.15.191:46372) 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a8eae4700 1 -- 172.21.15.191:0/2368537347 learned_addr learned my addr 172.21.15.191:0/2368537347 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a8e2e3700 1 -- 172.21.15.191:0/2368537347 >> v1:172.21.15.138:6789/0 conn(0x55b43aa40800 legacy=0x55b43ab2c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a8e2e3700 1 -- 172.21.15.191:0/2368537347 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa69d40 con 0x55b43aa40c00 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.075+0000 7f1a8dae2700 1 -- 172.21.15.191:0/2368537347 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147685037 0 0) 0x55b43b80e000 con 0x55b43aa40800 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a8e2e3700 1 --2- 172.21.15.191:0/2368537347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40c00 0x55b43ad28580 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55b43ad23890 tx=0x55b43ab31920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca357cadd26ade5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a8dae2700 1 -- 172.21.15.191:0/2368537347 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf94a0 con 0x55b43aa40c00 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a8dae2700 1 -- 172.21.15.191:0/2368537347 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43ad6b800 con 0x55b43aa40c00 2024-04-27T04:36:30.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a8dae2700 1 -- 172.21.15.191:0/2368537347 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf9600 con 0x55b43aa40c00 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2368537347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40c00 msgr2=0x55b43ad28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/2368537347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40c00 0x55b43ad28580 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55b43ad23890 tx=0x55b43ab31920 comp rx=0 tx=0).stop 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2368537347 shutdown_connections 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/2368537347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40c00 0x55b43ad28580 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2368537347 >> 172.21.15.191:0/2368537347 conn(0x55b43aa40400 msgr2=0x55b43ab2c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2368537347 shutdown_connections 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.076+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2368537347 wait complete. 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a90b3aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a90b3aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43aae2500 con 0x55b43aa40800 2024-04-27T04:36:30.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41328/0 (socket says 172.21.15.191:41328) 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.077+0000 7f1a8eae4700 1 -- 172.21.15.191:0/2027414088 learned_addr learned my addr 172.21.15.191:0/2027414088 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a8eae4700 1 -- 172.21.15.191:0/2027414088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa69ba0 con 0x55b43aa40800 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/2027414088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55b43b811770 tx=0x55b43ab31b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc843ec34a5b4fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a8d2e1700 1 -- 172.21.15.191:0/2027414088 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf91e0 con 0x55b43aa40800 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a8d2e1700 1 -- 172.21.15.191:0/2027414088 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43ad6bb00 con 0x55b43aa40800 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a8d2e1700 1 -- 172.21.15.191:0/2027414088 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf94a0 con 0x55b43aa40800 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2027414088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 msgr2=0x55b43ad28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/2027414088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55b43b811770 tx=0x55b43ab31b60 comp rx=0 tx=0).stop 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2027414088 shutdown_connections 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/2027414088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2027414088 >> 172.21.15.191:0/2027414088 conn(0x55b43aa40400 msgr2=0x55b43ab2d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2027414088 shutdown_connections 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.078+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2027414088 wait complete. 2024-04-27T04:36:30.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a90b3aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a90b3aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43aae23c0 con 0x55b43aa40800 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41332/0 (socket says 172.21.15.191:41332) 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a8eae4700 1 -- 172.21.15.191:0/3063600536 learned_addr learned my addr 172.21.15.191:0/3063600536 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a8eae4700 1 -- 172.21.15.191:0/3063600536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa69a00 con 0x55b43aa40800 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.079+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55b43b811e30 tx=0x55b43ab31a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a8bade700 1 -- 172.21.15.191:0/3063600536 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf9600 con 0x55b43aa40800 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a8bade700 1 -- 172.21.15.191:0/3063600536 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43aae1c80 con 0x55b43aa40800 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a8bade700 1 -- 172.21.15.191:0/3063600536 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf91e0 con 0x55b43aa40800 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b43aa69860 con 0x55b43aa40800 2024-04-27T04:36:30.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43aa696c0 con 0x55b43aa40800 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a90b3aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a90b3aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43aae2000 con 0x55b43aa41400 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.080+0000 7f1a8bade700 1 -- 172.21.15.191:0/3063600536 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b43b7e2b00 con 0x55b43aa40800 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8f2e5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8f2e5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41336/0 (socket says 172.21.15.191:41336) 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8f2e5700 1 -- 172.21.15.191:0/2225467193 learned_addr learned my addr 172.21.15.191:0/2225467193 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8bade700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6000 0x55b43ad29080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8bade700 1 -- 172.21.15.191:0/3063600536 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55b43aabb180 con 0x55b43aa40800 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6000 0x55b43ad29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8f2e5700 1 -- 172.21.15.191:0/2225467193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa68000 con 0x55b43aa41400 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8f2e5700 1 --2- 172.21.15.191:0/2225467193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x55b43add4030 tx=0x55b43ad8ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55546548da6591e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6000 0x55b43ad29080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b43b8d6f30 tx=0x55b43ad8e240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a892d9700 1 -- 172.21.15.191:0/2225467193 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf9600 con 0x55b43aa41400 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a892d9700 1 -- 172.21.15.191:0/2225467193 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43ad6a300 con 0x55b43aa41400 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.081+0000 7f1a892d9700 1 -- 172.21.15.191:0/2225467193 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf91e0 con 0x55b43aa41400 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2225467193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 msgr2=0x55b43ad28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/2225467193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x55b43add4030 tx=0x55b43ad8ede0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2225467193 shutdown_connections 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/2225467193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2225467193 >> 172.21.15.191:0/2225467193 conn(0x55b43aa40c00 msgr2=0x55b43ab2d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2225467193 shutdown_connections 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/2225467193 wait complete. 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a90b3aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43aa7c640 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a8f2e5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a8f2e5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41350/0 (socket says 172.21.15.191:41350) 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a8f2e5700 1 -- 172.21.15.191:0/1815027352 learned_addr learned my addr 172.21.15.191:0/1815027352 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.082+0000 7f1a8f2e5700 1 -- 172.21.15.191:0/1815027352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa68ea0 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a8f2e5700 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55b43b8d64e0 tx=0x55b43ad8e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a87ad6700 1 -- 172.21.15.191:0/1815027352 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf9a20 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a87ad6700 1 -- 172.21.15.191:0/1815027352 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43aae1c80 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b43aa68820 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a87ad6700 1 -- 172.21.15.191:0/1815027352 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf9600 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43aa689c0 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.083+0000 7f1a87ad6700 1 -- 172.21.15.191:0/1815027352 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b43b7e2dc0 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a87ad6700 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6c00 0x55b43ad29600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a87ad6700 1 -- 172.21.15.191:0/1815027352 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55b43aabb340 con 0x55b43aa41400 2024-04-27T04:36:30.098 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a90b3aa80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a90b3aa80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a8e2e3700 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6c00 0x55b43ad29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a8e2e3700 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6c00 0x55b43ad29600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55b43ad23890 tx=0x55b43ab31800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a90b3aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a90b3aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43aa7c8c0 con 0x55b43aa41c00 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41360/0 (socket says 172.21.15.191:41360) 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a8eae4700 1 -- 172.21.15.191:0/1266043285 learned_addr learned my addr 172.21.15.191:0/1266043285 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.084+0000 7f1a8eae4700 1 -- 172.21.15.191:0/1266043285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa68b60 con 0x55b43aa41c00 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/1266043285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55b43b8d6180 tx=0x55b43ad8f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6ac15255c9cd49 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a75ab2700 1 -- 172.21.15.191:0/1266043285 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf94a0 con 0x55b43aa41c00 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a75ab2700 1 -- 172.21.15.191:0/1266043285 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43aae1c80 con 0x55b43aa41c00 2024-04-27T04:36:30.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a75ab2700 1 -- 172.21.15.191:0/1266043285 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf98c0 con 0x55b43aa41c00 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1266043285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 msgr2=0x55b43ad29b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1266043285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55b43b8d6180 tx=0x55b43ad8f0e0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1266043285 shutdown_connections 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1266043285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1266043285 >> 172.21.15.191:0/1266043285 conn(0x55b43aa41800 msgr2=0x55b43ab2e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1266043285 shutdown_connections 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1266043285 wait complete. 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 Processor -- start 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.085+0000 7f1a90b3aa80 1 -- start start 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a90b3aa80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a90b3aa80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43aa7cf00 con 0x55b43aa41c00 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a8eae4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41366/0 (socket says 172.21.15.191:41366) 2024-04-27T04:36:30.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a8eae4700 1 -- 172.21.15.191:0/1544534793 learned_addr learned my addr 172.21.15.191:0/1544534793 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a8eae4700 1 -- 172.21.15.191:0/1544534793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43aa68d00 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55b43ae214a0 tx=0x55b43ad8e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a742af700 1 -- 172.21.15.191:0/1544534793 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf94a0 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a742af700 1 -- 172.21.15.191:0/1544534793 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b43ae96000 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b43aa68340 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a742af700 1 -- 172.21.15.191:0/1544534793 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b43aaf98c0 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.086+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43ae9e000 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a742af700 1 -- 172.21.15.191:0/1544534793 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b43b7e3b80 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a722ab700 20 reqs_thread_entry: start 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a742af700 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43af0a000 0x55b43ad2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a742af700 1 -- 172.21.15.191:0/1544534793 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55b43aabb500 con 0x55b43aa41c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b43add7c00 0x55b43ad2a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55b43af0a400 con 0x55b43add7c00 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.087+0000 7f1a8e2e3700 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43af0a000 0x55b43ad2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a8f2e5700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b43add7c00 0x55b43ad2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a8e2e3700 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43af0a000 0x55b43ad2ac00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b43ae20360 tx=0x55b43ab31860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a8f2e5700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b43add7c00 0x55b43ad2a100 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a8f2e5700 1 -- 172.21.15.191:0/3063600536 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b43af12240 con 0x55b43add7c00 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55b43af0a800 con 0x55b43add7c00 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a8f2e5700 1 -- 172.21.15.191:0/3063600536 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b43af12240 con 0x55b43add7c00 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b43af0ac00 0x55b43ad2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.088+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55b43af0b000 con 0x55b43af0ac00 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.089+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b43af0ac00 0x55b43ad2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.089+0000 7f1a8eae4700 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b43af0ac00 0x55b43ad2a680 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.089+0000 7f1a8eae4700 1 -- 172.21.15.191:0/3063600536 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x55b43af12900 con 0x55b43af0ac00 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6c00 msgr2=0x55b43ad29600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6c00 0x55b43ad29600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55b43ad23890 tx=0x55b43ab31800 comp rx=0 tx=0).stop 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 msgr2=0x55b43ad28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55b43b8d64e0 tx=0x55b43ad8e840 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 shutdown_connections 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6c00 0x55b43ad29600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1815027352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41400 0x55b43ad28b00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 >> 172.21.15.191:0/1815027352 conn(0x55b43aa40c00 msgr2=0x55b43ab2d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 shutdown_connections 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1815027352 wait complete. 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43af0a000 msgr2=0x55b43ad2ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43af0a000 0x55b43ad2ac00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b43ae20360 tx=0x55b43ab31860 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 msgr2=0x55b43ad29b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55b43ae214a0 tx=0x55b43ad8e1e0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 shutdown_connections 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43af0a000 0x55b43ad2ac00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/1544534793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa41c00 0x55b43ad29b80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 >> 172.21.15.191:0/1544534793 conn(0x55b43aa41800 msgr2=0x55b43ab2e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 shutdown_connections 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/1544534793 wait complete. 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b43add7c00 msgr2=0x55b43ad2a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b43add7c00 0x55b43ad2a100 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b43af0ac00 msgr2=0x55b43ad2a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b43af0ac00 0x55b43ad2a680 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6000 msgr2=0x55b43ad29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6000 0x55b43ad29080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b43b8d6f30 tx=0x55b43ad8e240 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 msgr2=0x55b43ad28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.090+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55b43b811e30 tx=0x55b43ab31a40 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 shutdown_connections 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55b43af0ac00 0x55b43ad2a680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55b43add7c00 0x55b43ad2a100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b43aa40800 0x55b43ad28580 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 --2- 172.21.15.191:0/3063600536 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55b43add6000 0x55b43ad29080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 >> 172.21.15.191:0/3063600536 conn(0x55b43aa40400 msgr2=0x55b43ab2d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 shutdown_connections 2024-04-27T04:36:30.105 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.091+0000 7f1a90b3aa80 1 -- 172.21.15.191:0/3063600536 wait complete. 2024-04-27T04:36:30.105 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "control_pool": "default.rgw.control", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "log_pool": "default.rgw.log", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T04:36:30.106 INFO:teuthology.orchestra.run.smithi191.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "system_key": { 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "", 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "" 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_pools": [ 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "default-placement", 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T04:36:30.107 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": true 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "new-placement", 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": ".rgw.buckets.index.2", 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": ".rgw.buckets.2" 2024-04-27T04:36:30.108 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": false 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: "realm_id": "", 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T04:36:30.109 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:30.109 DEBUG:tasks.util.rgw: json result: {'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T04:36:30.109 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T04:36:30.110 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T04:36:30.110 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-27T04:36:30.192 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:30.193 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.191+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.191+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e7d4ba80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55c1f56c4c00 0x55c1f59ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e7d4ba80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c1f59ef500 con 0x55c1f56c4800 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e7d4ba80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c1f5766a00 con 0x55c1f56c4c00 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e54f4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55c1f56c4c00 0x55c1f59ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e5cf5700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55c1f56c4800 0x55c1f57b0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46374/0 (socket says 172.21.15.191:46374) 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/780872565 learned_addr learned my addr 172.21.15.191:0/780872565 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e4cf3700 1 -- 172.21.15.191:0/780872565 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014015980 0 0) 0x55c1f6492000 con 0x55c1f56c4800 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e4cf3700 1 -- 172.21.15.191:0/780872565 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55c1f59ef680 con 0x55c1f56c4800 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e54f4700 1 -- 172.21.15.191:0/780872565 >> v1:172.21.15.138:6789/0 conn(0x55c1f56c4800 legacy=0x55c1f57b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.192+0000 7fe6e54f4700 1 -- 172.21.15.191:0/780872565 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56edd40 con 0x55c1f56c4c00 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e54f4700 1 --2- 172.21.15.191:0/780872565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4c00 0x55c1f59ac580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55c1f59a7890 tx=0x55c1f57b5980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc97ae4ea20ea512 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e4cf3700 1 -- 172.21.15.191:0/780872565 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d4a0 con 0x55c1f56c4c00 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e4cf3700 1 -- 172.21.15.191:0/780872565 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f59ef500 con 0x55c1f56c4c00 2024-04-27T04:36:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e4cf3700 1 -- 172.21.15.191:0/780872565 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d600 con 0x55c1f56c4c00 2024-04-27T04:36:30.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/780872565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4c00 msgr2=0x55c1f59ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/780872565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4c00 0x55c1f59ac580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55c1f59a7890 tx=0x55c1f57b5980 comp rx=0 tx=0).stop 2024-04-27T04:36:30.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/780872565 shutdown_connections 2024-04-27T04:36:30.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/780872565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4c00 0x55c1f59ac580 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/780872565 >> 172.21.15.191:0/780872565 conn(0x55c1f56c4400 msgr2=0x55c1f57b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/780872565 shutdown_connections 2024-04-27T04:36:30.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.193+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/780872565 wait complete. 2024-04-27T04:36:30.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e7d4ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e7d4ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5766500 con 0x55c1f56c4800 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41384/0 (socket says 172.21.15.191:41384) 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.194+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/3203213492 learned_addr learned my addr 172.21.15.191:0/3203213492 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/3203213492 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56edba0 con 0x55c1f56c4800 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/3203213492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55c1f6495d70 tx=0x55c1f57b5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7557445d13371a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e44f2700 1 -- 172.21.15.191:0/3203213492 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d4a0 con 0x55c1f56c4800 2024-04-27T04:36:30.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e44f2700 1 -- 172.21.15.191:0/3203213492 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f59efb00 con 0x55c1f56c4800 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e44f2700 1 -- 172.21.15.191:0/3203213492 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d600 con 0x55c1f56c4800 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3203213492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 msgr2=0x55c1f59acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3203213492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55c1f6495d70 tx=0x55c1f57b5b60 comp rx=0 tx=0).stop 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3203213492 shutdown_connections 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3203213492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3203213492 >> 172.21.15.191:0/3203213492 conn(0x55c1f56c4400 msgr2=0x55c1f57b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3203213492 shutdown_connections 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.195+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3203213492 wait complete. 2024-04-27T04:36:30.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e7d4ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e7d4ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f57663c0 con 0x55c1f56c4800 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41388/0 (socket says 172.21.15.191:41388) 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/2673309950 learned_addr learned my addr 172.21.15.191:0/2673309950 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/2673309950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56eda00 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.196+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55c1f6495e00 tx=0x55c1f57b5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e2cef700 1 -- 172.21.15.191:0/2673309950 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d8c0 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e2cef700 1 -- 172.21.15.191:0/2673309950 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f5765c80 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e2cef700 1 -- 172.21.15.191:0/2673309950 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d4a0 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c1f56ed860 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f56ed6c0 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e2cef700 1 -- 172.21.15.191:0/2673309950 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c1f6466b00 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e2cef700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a52000 0x55c1f59ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e2cef700 1 -- 172.21.15.191:0/2673309950 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c1f573f180 con 0x55c1f56c4800 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e7d4ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.197+0000 7fe6e7d4ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5766000 con 0x55c1f56c5800 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e64f6700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a52000 0x55c1f59ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41390/0 (socket says 172.21.15.191:41390) 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/1209690939 learned_addr learned my addr 172.21.15.191:0/1209690939 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e64f6700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a52000 0x55c1f59ad080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c1f6495dd0 tx=0x55c1f5a0cde0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/1209690939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56ec000 con 0x55c1f56c5800 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/1209690939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55c1f6552270 tx=0x55c1f5a0c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c07e6df4571e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e04ea700 1 -- 172.21.15.191:0/1209690939 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d600 con 0x55c1f56c5800 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e04ea700 1 -- 172.21.15.191:0/1209690939 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f5765e00 con 0x55c1f56c5800 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.198+0000 7fe6e04ea700 1 -- 172.21.15.191:0/1209690939 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577da20 con 0x55c1f56c5800 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/1209690939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 msgr2=0x55c1f59ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/1209690939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55c1f6552270 tx=0x55c1f5a0c240 comp rx=0 tx=0).stop 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/1209690939 shutdown_connections 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/1209690939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/1209690939 >> 172.21.15.191:0/1209690939 conn(0x55c1f56c5400 msgr2=0x55c1f57b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/1209690939 shutdown_connections 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/1209690939 wait complete. 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e7d4ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5700640 con 0x55c1f56c5800 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41402/0 (socket says 172.21.15.191:41402) 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/2450349228 learned_addr learned my addr 172.21.15.191:0/2450349228 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.199+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/2450349228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56ecea0 con 0x55c1f56c5800 2024-04-27T04:36:30.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a4ee40 tx=0x55c1f5a0cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6dece7700 1 -- 172.21.15.191:0/2450349228 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d600 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6dece7700 1 -- 172.21.15.191:0/2450349228 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f5a7c000 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c1f56ec820 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6dece7700 1 -- 172.21.15.191:0/2450349228 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577da20 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f56ec9c0 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.200+0000 7fe6dece7700 1 -- 172.21.15.191:0/2450349228 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c1f6466dc0 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6dece7700 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a53000 0x55c1f59ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e7d4ba80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e7d4ba80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6dece7700 1 -- 172.21.15.191:0/2450349228 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c1f573f340 con 0x55c1f56c5800 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e7d4ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e7d4ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f57008c0 con 0x55c1f5a53400 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41410/0 (socket says 172.21.15.191:41410) 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.201+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/3535778078 learned_addr learned my addr 172.21.15.191:0/3535778078 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/3535778078 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56ecb60 con 0x55c1f5a53400 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e54f4700 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a53000 0x55c1f59ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/3535778078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a8e150 tx=0x55c1f5a0d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f28f1de35d2291 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6cccc3700 1 -- 172.21.15.191:0/3535778078 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d8c0 con 0x55c1f5a53400 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6cccc3700 1 -- 172.21.15.191:0/3535778078 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f5a7c300 con 0x55c1f5a53400 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6cccc3700 1 -- 172.21.15.191:0/3535778078 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d4a0 con 0x55c1f5a53400 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3535778078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 msgr2=0x55c1f59adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3535778078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a8e150 tx=0x55c1f5a0d560 comp rx=0 tx=0).stop 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e54f4700 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a53000 0x55c1f59ae680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55c1f59a7a40 tx=0x55c1f57b5800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3535778078 shutdown_connections 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3535778078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3535778078 >> 172.21.15.191:0/3535778078 conn(0x55c1f56c5c00 msgr2=0x55c1f57b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3535778078 shutdown_connections 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.202+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3535778078 wait complete. 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e7d4ba80 1 Processor -- start 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e7d4ba80 1 -- start start 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e7d4ba80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e7d4ba80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5700f00 con 0x55c1f5a53400 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.218 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e5cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41414/0 (socket says 172.21.15.191:41414) 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/3276344466 learned_addr learned my addr 172.21.15.191:0/3276344466 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/3276344466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f56ecd00 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a8f470 tx=0x55c1f57b5f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6cb4c0700 1 -- 172.21.15.191:0/3276344466 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d8c0 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c1f56ec340 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.203+0000 7fe6cb4c0700 1 -- 172.21.15.191:0/3276344466 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1f5a7c480 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6cb4c0700 1 -- 172.21.15.191:0/3276344466 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c1f577d4a0 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f5af4000 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6cb4c0700 1 -- 172.21.15.191:0/3276344466 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c1f6467080 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6cb4c0700 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5b58000 0x55c1f59aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6cb4c0700 1 -- 172.21.15.191:0/3276344466 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c1f573f500 con 0x55c1f5a53400 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.204+0000 7fe6e54f4700 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5b58000 0x55c1f59aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.205+0000 7fe6c94bc700 20 reqs_thread_entry: start 2024-04-27T04:36:30.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.205+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c1f5a53c00 0x55c1f59ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.205+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c1f5b58400 con 0x55c1f5a53c00 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.205+0000 7fe6e54f4700 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5b58000 0x55c1f59aec00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c1f59a7890 tx=0x55c1f57b5860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.205+0000 7fe6e64f6700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c1f5a53c00 0x55c1f59ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.205+0000 7fe6e64f6700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c1f5a53c00 0x55c1f59ae100 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e64f6700 1 -- 172.21.15.191:0/2673309950 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c1f5b62240 con 0x55c1f5a53c00 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c1f5b58800 0x55c1f59af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:2 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c1f5b58c00 con 0x55c1f5b58800 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c1f5b58800 0x55c1f59af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e5cf5700 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c1f5b58800 0x55c1f59af180 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/2673309950 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(2 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x55c1f5b62480 con 0x55c1f5b58800 2024-04-27T04:36:30.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.206+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c1f5b59000 con 0x55c1f5b58800 2024-04-27T04:36:30.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e5cf5700 1 -- 172.21.15.191:0/2673309950 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(3 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [stat,call,call,writefull 0~874] v33'4 uv4 ondisk = 0) v8 ==== 316+0+0 (crc 0 0 0) 0x55c1f5b62480 con 0x55c1f5b58800 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a53000 msgr2=0x55c1f59ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a53000 0x55c1f59ae680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55c1f59a7a40 tx=0x55c1f57b5800 comp rx=0 tx=0).stop 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 msgr2=0x55c1f59ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a4ee40 tx=0x55c1f5a0cfc0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 shutdown_connections 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a53000 0x55c1f59ae680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2450349228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c5800 0x55c1f59ad600 secure :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a4ee40 tx=0x55c1f5a0cfc0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 >> 172.21.15.191:0/2450349228 conn(0x55c1f56c5400 msgr2=0x55c1f57b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 shutdown_connections 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.208+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2450349228 wait complete. 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5b58000 msgr2=0x55c1f59aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5b58000 0x55c1f59aec00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c1f59a7890 tx=0x55c1f57b5860 comp rx=0 tx=0).stop 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 msgr2=0x55c1f59adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x55c1f5a8f470 tx=0x55c1f57b5f80 comp rx=0 tx=0).stop 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 shutdown_connections 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5b58000 0x55c1f59aec00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/3276344466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f5a53400 0x55c1f59adb80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 >> 172.21.15.191:0/3276344466 conn(0x55c1f56c5c00 msgr2=0x55c1f57b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 shutdown_connections 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/3276344466 wait complete. 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c1f5a53c00 msgr2=0x55c1f59ae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c1f5a53c00 0x55c1f59ae100 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.225 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c1f5b58800 msgr2=0x55c1f59af180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c1f5b58800 0x55c1f59af180 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a52000 msgr2=0x55c1f59ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a52000 0x55c1f59ad080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c1f6495dd0 tx=0x55c1f5a0cde0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 msgr2=0x55c1f59acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55c1f6495e00 tx=0x55c1f57b5920 comp rx=0 tx=0).stop 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 shutdown_connections 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55c1f5b58800 0x55c1f59af180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55c1f5a53c00 0x55c1f59ae100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1f56c4800 0x55c1f59acb00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 --2- 172.21.15.191:0/2673309950 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55c1f5a52000 0x55c1f59ad080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 >> 172.21.15.191:0/2673309950 conn(0x55c1f56c4400 msgr2=0x55c1f57b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 shutdown_connections 2024-04-27T04:36:30.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.209+0000 7fe6e7d4ba80 1 -- 172.21.15.191:0/2673309950 wait complete. 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "control_pool": "default.rgw.control", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "log_pool": "default.rgw.log", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T04:36:30.228 INFO:teuthology.orchestra.run.smithi191.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "system_key": { 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "", 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "" 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_pools": [ 2024-04-27T04:36:30.229 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "key": "default-placement", 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "val": { 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": { 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD": { 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.230 INFO:teuthology.orchestra.run.smithi191.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: "index_type": 0, 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: "inline_data": true 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: "realm_id": "", 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T04:36:30.231 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:30.231 DEBUG:tasks.util.rgw: json result: {'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T04:36:30.231 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-04-27T04:36:30.232 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-04-27T04:36:30.232 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-04-27T04:36:30.276 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:30.276 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ae29da80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55fe397f2c00 0x55fe39ada580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ae29da80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fe39b1d500 con 0x55fe397f2800 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ae29da80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fe39894a00 con 0x55fe397f2c00 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5aba46700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55fe397f2c00 0x55fe39ada580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ac247700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55fe397f2800 0x55fe398de800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46390/0 (socket says 172.21.15.191:46390) 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.275+0000 7fc5ac247700 1 -- 172.21.15.191:0/4029433594 learned_addr learned my addr 172.21.15.191:0/4029433594 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:30.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5aba46700 1 -- 172.21.15.191:0/4029433594 >> v1:172.21.15.138:6789/0 conn(0x55fe397f2800 legacy=0x55fe398de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5aba46700 1 -- 172.21.15.191:0/4029433594 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981bd40 con 0x55fe397f2c00 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ab245700 1 -- 172.21.15.191:0/4029433594 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587349779 0 0) 0x55fe3a5c0000 con 0x55fe397f2800 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5aba46700 1 --2- 172.21.15.191:0/4029433594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2c00 0x55fe39ada580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55fe39ad5890 tx=0x55fe398e3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b910afe50a52d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ab245700 1 -- 172.21.15.191:0/4029433594 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab4a0 con 0x55fe397f2c00 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ab245700 1 -- 172.21.15.191:0/4029433594 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39b1d800 con 0x55fe397f2c00 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ab245700 1 -- 172.21.15.191:0/4029433594 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab600 con 0x55fe397f2c00 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ae29da80 1 -- 172.21.15.191:0/4029433594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2c00 msgr2=0x55fe39ada580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/4029433594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2c00 0x55fe39ada580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55fe39ad5890 tx=0x55fe398e3920 comp rx=0 tx=0).stop 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ae29da80 1 -- 172.21.15.191:0/4029433594 shutdown_connections 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/4029433594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2c00 0x55fe39ada580 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.276+0000 7fc5ae29da80 1 -- 172.21.15.191:0/4029433594 >> 172.21.15.191:0/4029433594 conn(0x55fe397f2400 msgr2=0x55fe398de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ae29da80 1 -- 172.21.15.191:0/4029433594 shutdown_connections 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ae29da80 1 -- 172.21.15.191:0/4029433594 wait complete. 2024-04-27T04:36:30.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ae29da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ae29da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe39894500 con 0x55fe397f2800 2024-04-27T04:36:30.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.277+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41428/0 (socket says 172.21.15.191:41428) 2024-04-27T04:36:30.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5ac247700 1 -- 172.21.15.191:0/2079141164 learned_addr learned my addr 172.21.15.191:0/2079141164 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5ac247700 1 -- 172.21.15.191:0/2079141164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981bba0 con 0x55fe397f2800 2024-04-27T04:36:30.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5ac247700 1 --2- 172.21.15.191:0/2079141164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55fe3a5c3770 tx=0x55fe398e3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0c3047643e4d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5aaa44700 1 -- 172.21.15.191:0/2079141164 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab1e0 con 0x55fe397f2800 2024-04-27T04:36:30.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5aaa44700 1 -- 172.21.15.191:0/2079141164 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39b1db00 con 0x55fe397f2800 2024-04-27T04:36:30.292 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5aaa44700 1 -- 172.21.15.191:0/2079141164 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab4a0 con 0x55fe397f2800 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5ae29da80 1 -- 172.21.15.191:0/2079141164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 msgr2=0x55fe39ada580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.278+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/2079141164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55fe3a5c3770 tx=0x55fe398e3b60 comp rx=0 tx=0).stop 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 -- 172.21.15.191:0/2079141164 shutdown_connections 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/2079141164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 -- 172.21.15.191:0/2079141164 >> 172.21.15.191:0/2079141164 conn(0x55fe397f2400 msgr2=0x55fe398df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 -- 172.21.15.191:0/2079141164 shutdown_connections 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 -- 172.21.15.191:0/2079141164 wait complete. 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ae29da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe398943c0 con 0x55fe397f2800 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41432/0 (socket says 172.21.15.191:41432) 2024-04-27T04:36:30.293 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.279+0000 7fc5ac247700 1 -- 172.21.15.191:0/1292711663 learned_addr learned my addr 172.21.15.191:0/1292711663 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5ac247700 1 -- 172.21.15.191:0/1292711663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981ba00 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5ac247700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55fe3a5c3e30 tx=0x55fe398e3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5a9241700 1 -- 172.21.15.191:0/1292711663 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab600 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5a9241700 1 -- 172.21.15.191:0/1292711663 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39893c80 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe3981b860 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5a9241700 1 -- 172.21.15.191:0/1292711663 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab1e0 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.280+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe3981b6c0 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5a9241700 1 -- 172.21.15.191:0/1292711663 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fe3a594b00 con 0x55fe397f2800 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5ae29da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5ae29da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe39894000 con 0x55fe397f3400 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5a9241700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84000 0x55fe39adb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5aca48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5a9241700 1 -- 172.21.15.191:0/1292711663 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55fe3986d180 con 0x55fe397f2800 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5aca48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41436/0 (socket says 172.21.15.191:41436) 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5aca48700 1 -- 172.21.15.191:0/199201186 learned_addr learned my addr 172.21.15.191:0/199201186 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5ac247700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84000 0x55fe39adb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5aca48700 1 -- 172.21.15.191:0/199201186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981a000 con 0x55fe397f3400 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5ac247700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84000 0x55fe39adb080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55fe3a684f00 tx=0x55fe39b3e240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5aca48700 1 --2- 172.21.15.191:0/199201186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55fe3a5c3e90 tx=0x55fe39b3ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e609138da3a4cd7f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5a6a3c700 1 -- 172.21.15.191:0/199201186 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab600 con 0x55fe397f3400 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5a6a3c700 1 -- 172.21.15.191:0/199201186 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39b1c300 con 0x55fe397f3400 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.281+0000 7fc5a6a3c700 1 -- 172.21.15.191:0/199201186 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab1e0 con 0x55fe397f3400 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- 172.21.15.191:0/199201186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 msgr2=0x55fe39adab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/199201186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55fe3a5c3e90 tx=0x55fe39b3ede0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- 172.21.15.191:0/199201186 shutdown_connections 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/199201186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- 172.21.15.191:0/199201186 >> 172.21.15.191:0/199201186 conn(0x55fe397f2c00 msgr2=0x55fe398df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- 172.21.15.191:0/199201186 shutdown_connections 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- 172.21.15.191:0/199201186 wait complete. 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.282+0000 7fc5ae29da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3982e640 con 0x55fe397f3400 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5aca48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5aca48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41452/0 (socket says 172.21.15.191:41452) 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5aca48700 1 -- 172.21.15.191:0/1231705189 learned_addr learned my addr 172.21.15.191:0/1231705189 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5aca48700 1 -- 172.21.15.191:0/1231705189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981aea0 con 0x55fe397f3400 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5aca48700 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55fe3a684450 tx=0x55fe398e3f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5a5239700 1 -- 172.21.15.191:0/1231705189 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398aba20 con 0x55fe397f3400 2024-04-27T04:36:30.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5a5239700 1 -- 172.21.15.191:0/1231705189 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39893c80 con 0x55fe397f3400 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe3981a820 con 0x55fe397f3400 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5a5239700 1 -- 172.21.15.191:0/1231705189 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab600 con 0x55fe397f3400 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.283+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe3981a9c0 con 0x55fe397f3400 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.284+0000 7fc5a5239700 1 -- 172.21.15.191:0/1231705189 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fe3a594dc0 con 0x55fe397f3400 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.284+0000 7fc5a5239700 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84c00 0x55fe39adb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.284+0000 7fc5a5239700 1 -- 172.21.15.191:0/1231705189 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55fe3986d340 con 0x55fe397f3400 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.284+0000 7fc5aba46700 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84c00 0x55fe39adb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.284+0000 7fc5aba46700 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84c00 0x55fe39adb600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55fe39ad5890 tx=0x55fe398e3800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ae29da80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ae29da80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.297 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ae29da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ae29da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3982e8c0 con 0x55fe397f3c00 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41466/0 (socket says 172.21.15.191:41466) 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ac247700 1 -- 172.21.15.191:0/822897268 learned_addr learned my addr 172.21.15.191:0/822897268 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.285+0000 7fc5ac247700 1 -- 172.21.15.191:0/822897268 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981ab60 con 0x55fe397f3c00 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ac247700 1 --2- 172.21.15.191:0/822897268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x55fe3a684180 tx=0x55fe39b3f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5422a1280a7b6a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc593215700 1 -- 172.21.15.191:0/822897268 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab4a0 con 0x55fe397f3c00 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc593215700 1 -- 172.21.15.191:0/822897268 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39893c80 con 0x55fe397f3c00 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc593215700 1 -- 172.21.15.191:0/822897268 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab8c0 con 0x55fe397f3c00 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 -- 172.21.15.191:0/822897268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 msgr2=0x55fe39adbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/822897268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x55fe3a684180 tx=0x55fe39b3f0e0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 -- 172.21.15.191:0/822897268 shutdown_connections 2024-04-27T04:36:30.298 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/822897268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 -- 172.21.15.191:0/822897268 >> 172.21.15.191:0/822897268 conn(0x55fe397f3800 msgr2=0x55fe398e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 -- 172.21.15.191:0/822897268 shutdown_connections 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 -- 172.21.15.191:0/822897268 wait complete. 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 Processor -- start 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.286+0000 7fc5ae29da80 1 -- start start 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ae29da80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ae29da80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3982ef00 con 0x55fe397f3c00 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ac247700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41476/0 (socket says 172.21.15.191:41476) 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ac247700 1 -- 172.21.15.191:0/19099865 learned_addr learned my addr 172.21.15.191:0/19099865 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ac247700 1 -- 172.21.15.191:0/19099865 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3981ad00 con 0x55fe397f3c00 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ac247700 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x55fe39bdd140 tx=0x55fe39b3e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc591a12700 1 -- 172.21.15.191:0/19099865 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab4a0 con 0x55fe397f3c00 2024-04-27T04:36:30.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc591a12700 1 -- 172.21.15.191:0/19099865 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe39c46000 con 0x55fe397f3c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe3981a340 con 0x55fe397f3c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc591a12700 1 -- 172.21.15.191:0/19099865 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fe398ab8c0 con 0x55fe397f3c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.287+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe39c4e000 con 0x55fe397f3c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.288+0000 7fc591a12700 1 -- 172.21.15.191:0/19099865 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fe3a595b80 con 0x55fe397f3c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.288+0000 7fc591a12700 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39ca2000 0x55fe39adcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.288+0000 7fc591a12700 1 -- 172.21.15.191:0/19099865 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55fe3986d500 con 0x55fe397f3c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.288+0000 7fc5aba46700 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39ca2000 0x55fe39adcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5aba46700 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39ca2000 0x55fe39adcc00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55fe39b83530 tx=0x55fe398e3860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc58fa0e700 20 reqs_thread_entry: start 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fe39b85c00 0x55fe39adc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55fe39ca2400 con 0x55fe39b85c00 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5aca48700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fe39b85c00 0x55fe39adc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5aca48700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fe39b85c00 0x55fe39adc100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5aca48700 1 -- 172.21.15.191:0/1292711663 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fe39cb0240 con 0x55fe39b85c00 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.289+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55fe39ca2800 con 0x55fe39b85c00 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.290+0000 7fc5aca48700 1 -- 172.21.15.191:0/1292711663 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55fe39cb0240 con 0x55fe39b85c00 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.290+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fe39ca2c00 0x55fe39adc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.290+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:3 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55fe39ca3000 con 0x55fe39ca2c00 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.290+0000 7fc5ac247700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fe39ca2c00 0x55fe39adc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.290+0000 7fc5ac247700 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fe39ca2c00 0x55fe39adc680 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.290+0000 7fc5ac247700 1 -- 172.21.15.191:0/1292711663 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(3 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [call out=24b,read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+444 (crc 0 0 0) 0x55fe39cb0900 con 0x55fe39ca2c00 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84c00 msgr2=0x55fe39adb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84c00 0x55fe39adb600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55fe39ad5890 tx=0x55fe398e3800 comp rx=0 tx=0).stop 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 msgr2=0x55fe39adab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55fe3a684450 tx=0x55fe398e3f20 comp rx=0 tx=0).stop 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 shutdown_connections 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84c00 0x55fe39adb600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1231705189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3400 0x55fe39adab00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 >> 172.21.15.191:0/1231705189 conn(0x55fe397f2c00 msgr2=0x55fe398df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 shutdown_connections 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1231705189 wait complete. 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39ca2000 msgr2=0x55fe39adcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39ca2000 0x55fe39adcc00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55fe39b83530 tx=0x55fe398e3860 comp rx=0 tx=0).stop 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 msgr2=0x55fe39adbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x55fe39bdd140 tx=0x55fe39b3e1e0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 shutdown_connections 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39ca2000 0x55fe39adcc00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/19099865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f3c00 0x55fe39adbb80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 >> 172.21.15.191:0/19099865 conn(0x55fe397f3800 msgr2=0x55fe398e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 shutdown_connections 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.291+0000 7fc5ae29da80 1 -- 172.21.15.191:0/19099865 wait complete. 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fe39b85c00 msgr2=0x55fe39adc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fe39b85c00 0x55fe39adc100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fe39ca2c00 msgr2=0x55fe39adc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fe39ca2c00 0x55fe39adc680 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84000 msgr2=0x55fe39adb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84000 0x55fe39adb080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55fe3a684f00 tx=0x55fe39b3e240 comp rx=0 tx=0).stop 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 msgr2=0x55fe39ada580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55fe3a5c3e30 tx=0x55fe398e3a40 comp rx=0 tx=0).stop 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 shutdown_connections 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55fe39ca2c00 0x55fe39adc680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55fe39b85c00 0x55fe39adc100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe397f2800 0x55fe39ada580 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 --2- 172.21.15.191:0/1292711663 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55fe39b84000 0x55fe39adb080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 >> 172.21.15.191:0/1292711663 conn(0x55fe397f2400 msgr2=0x55fe398df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 shutdown_connections 2024-04-27T04:36:30.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.292+0000 7fc5ae29da80 1 -- 172.21.15.191:0/1292711663 wait complete. 2024-04-27T04:36:30.307 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:30.307 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "29d2c105-c074-4639-b257-47b1060771f9", 2024-04-27T04:36:30.307 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:36:30.307 INFO:teuthology.orchestra.run.smithi191.stdout: "api_name": "default", 2024-04-27T04:36:30.307 INFO:teuthology.orchestra.run.smithi191.stdout: "is_master": true, 2024-04-27T04:36:30.307 INFO:teuthology.orchestra.run.smithi191.stdout: "endpoints": [], 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "hostnames": [], 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "hostnames_s3website": [], 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "master_zone": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "zones": [ 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "id": "e734f6f6-f5d6-4534-ad88-d40773f7ed1b", 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default", 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "endpoints": [], 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "log_meta": false, 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "log_data": false, 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_index_max_shards": 11, 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "read_only": false, 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "tier_type": "", 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "sync_from_all": true, 2024-04-27T04:36:30.308 INFO:teuthology.orchestra.run.smithi191.stdout: "sync_from": [], 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "redirect_zone": "", 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "supported_features": [ 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "compress-encrypted", 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "resharding" 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_targets": [ 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "name": "default-placement", 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "tags": [], 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "storage_classes": [ 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: "STANDARD" 2024-04-27T04:36:30.309 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "default-placement", 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: "realm_id": "", 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: "sync_policy": { 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: "groups": [] 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled_features": [ 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: "resharding" 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-04-27T04:36:30.310 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:30.310 DEBUG:tasks.util.rgw: json result: {'id': '29d2c105-c074-4639-b257-47b1060771f9', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'zones': [{'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['compress-encrypted', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['resharding']} 2024-04-27T04:36:30.313 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-04-27T04:36:30.330 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-27T04:36:30.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-27T04:36:30.331 DEBUG:teuthology.orchestra.run.smithi191:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-04-27T04:36:30.386 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setuser ceph since I am not root 2024-04-27T04:36:30.386 INFO:teuthology.orchestra.run.smithi191.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T04:36:30.403 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f84bca80 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f79cdfac00 0x55f79d0e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f84bca80 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f79d125500 con 0x55f79cdfa800 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f84bca80 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f79ce9ca00 con 0x55f79cdfac00 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f5c65700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x55f79cdfac00 0x55f79d0e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f6466700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x55f79cdfa800 0x55f79cee6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.191:46406/0 (socket says 172.21.15.191:46406) 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f6466700 1 -- 172.21.15.191:0/1654377721 learned_addr learned my addr 172.21.15.191:0/1654377721 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f5c65700 1 -- 172.21.15.191:0/1654377721 >> v1:172.21.15.138:6789/0 conn(0x55f79cdfa800 legacy=0x55f79cee6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f5c65700 1 -- 172.21.15.191:0/1654377721 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce23d40 con 0x55f79cdfac00 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.385+0000 7f28f5464700 1 -- 172.21.15.191:0/1654377721 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097076534 0 0) 0x55f79dbc8000 con 0x55f79cdfa800 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f5c65700 1 --2- 172.21.15.191:0/1654377721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfac00 0x55f79d0e2580 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x55f79d0dd890 tx=0x55f79ceeb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4ea5b9c38c9107 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f5464700 1 -- 172.21.15.191:0/1654377721 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb34a0 con 0x55f79cdfac00 2024-04-27T04:36:30.404 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f5464700 1 -- 172.21.15.191:0/1654377721 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79d125800 con 0x55f79cdfac00 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f5464700 1 -- 172.21.15.191:0/1654377721 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb3600 con 0x55f79cdfac00 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 -- 172.21.15.191:0/1654377721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfac00 msgr2=0x55f79d0e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 --2- 172.21.15.191:0/1654377721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfac00 0x55f79d0e2580 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x55f79d0dd890 tx=0x55f79ceeb920 comp rx=0 tx=0).stop 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 -- 172.21.15.191:0/1654377721 shutdown_connections 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 --2- 172.21.15.191:0/1654377721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfac00 0x55f79d0e2580 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 -- 172.21.15.191:0/1654377721 >> 172.21.15.191:0/1654377721 conn(0x55f79cdfa400 msgr2=0x55f79cee6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 -- 172.21.15.191:0/1654377721 shutdown_connections 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.386+0000 7f28f84bca80 1 -- 172.21.15.191:0/1654377721 wait complete. 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f84bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f84bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ce9c500 con 0x55f79cdfa800 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f6466700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f6466700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41488/0 (socket says 172.21.15.191:41488) 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.387+0000 7f28f6466700 1 -- 172.21.15.191:0/3429168990 learned_addr learned my addr 172.21.15.191:0/3429168990 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f6466700 1 -- 172.21.15.191:0/3429168990 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce23ba0 con 0x55f79cdfa800 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f6466700 1 --2- 172.21.15.191:0/3429168990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55f79dbcbd70 tx=0x55f79ceebb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a709ff546693a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f4c63700 1 -- 172.21.15.191:0/3429168990 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb31e0 con 0x55f79cdfa800 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f4c63700 1 -- 172.21.15.191:0/3429168990 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79d125b00 con 0x55f79cdfa800 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f4c63700 1 -- 172.21.15.191:0/3429168990 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb34a0 con 0x55f79cdfa800 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 -- 172.21.15.191:0/3429168990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 msgr2=0x55f79d0e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 --2- 172.21.15.191:0/3429168990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55f79dbcbd70 tx=0x55f79ceebb60 comp rx=0 tx=0).stop 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 -- 172.21.15.191:0/3429168990 shutdown_connections 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 --2- 172.21.15.191:0/3429168990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 -- 172.21.15.191:0/3429168990 >> 172.21.15.191:0/3429168990 conn(0x55f79cdfa400 msgr2=0x55f79cee7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 -- 172.21.15.191:0/3429168990 shutdown_connections 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.388+0000 7f28f84bca80 1 -- 172.21.15.191:0/3429168990 wait complete. 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f84bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f84bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ce9c3c0 con 0x55f79cdfa800 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f6466700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f6466700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41502/0 (socket says 172.21.15.191:41502) 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f6466700 1 -- 172.21.15.191:0/560341667 learned_addr learned my addr 172.21.15.191:0/560341667 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.389+0000 7f28f6466700 1 -- 172.21.15.191:0/560341667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce23a00 con 0x55f79cdfa800 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f6466700 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x55f79dbcbe60 tx=0x55f79ceeba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f3460700 1 -- 172.21.15.191:0/560341667 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb3600 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f3460700 1 -- 172.21.15.191:0/560341667 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79ce9bc80 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f79ce23860 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f3460700 1 -- 172.21.15.191:0/560341667 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb31e0 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f79ce236c0 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f3460700 1 -- 172.21.15.191:0/560341667 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f79db9cb00 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f84bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.390+0000 7f28f84bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ce36640 con 0x55f79cdfb400 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f3460700 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18a000 0x55f79d0e3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f3460700 1 -- 172.21.15.191:0/560341667 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f79ce75180 con 0x55f79cdfa800 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6c67700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6c67700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41514/0 (socket says 172.21.15.191:41514) 2024-04-27T04:36:30.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6c67700 1 -- 172.21.15.191:0/642538032 learned_addr learned my addr 172.21.15.191:0/642538032 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6466700 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18a000 0x55f79d0e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6c67700 1 -- 172.21.15.191:0/642538032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce22000 con 0x55f79cdfb400 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6466700 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18a000 0x55f79d0e3080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55f79dc8cf00 tx=0x55f79d146240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f6c67700 1 --2- 172.21.15.191:0/642538032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55f79dbcbf50 tx=0x55f79d146de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d465cbef7cdf27c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f0c5b700 1 -- 172.21.15.191:0/642538032 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb3600 con 0x55f79cdfb400 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f0c5b700 1 -- 172.21.15.191:0/642538032 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79d124300 con 0x55f79cdfb400 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.391+0000 7f28f0c5b700 1 -- 172.21.15.191:0/642538032 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb31e0 con 0x55f79cdfb400 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- 172.21.15.191:0/642538032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 msgr2=0x55f79d0e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 --2- 172.21.15.191:0/642538032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55f79dbcbf50 tx=0x55f79d146de0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- 172.21.15.191:0/642538032 shutdown_connections 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 --2- 172.21.15.191:0/642538032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- 172.21.15.191:0/642538032 >> 172.21.15.191:0/642538032 conn(0x55f79cdfac00 msgr2=0x55f79cee7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.409 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- 172.21.15.191:0/642538032 shutdown_connections 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- 172.21.15.191:0/642538032 wait complete. 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f84bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ce368c0 con 0x55f79cdfb400 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f6c67700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f6c67700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41524/0 (socket says 172.21.15.191:41524) 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 learned_addr learned my addr 172.21.15.191:0/339926555 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.392+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce22ea0 con 0x55f79cdfb400 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.393+0000 7f28f6c67700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55f79dc8c420 tx=0x55f79ceebf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.393+0000 7f28ef458700 1 -- 172.21.15.191:0/339926555 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb3a20 con 0x55f79cdfb400 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.393+0000 7f28ef458700 1 -- 172.21.15.191:0/339926555 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79ce9bc80 con 0x55f79cdfb400 2024-04-27T04:36:30.410 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.393+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f79ce22820 con 0x55f79cdfb400 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.393+0000 7f28ef458700 1 -- 172.21.15.191:0/339926555 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb3600 con 0x55f79cdfb400 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.393+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f79ce229c0 con 0x55f79cdfb400 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28ef458700 1 -- 172.21.15.191:0/339926555 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f79db9cdc0 con 0x55f79cdfb400 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28ef458700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18ac00 0x55f79d0e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28ef458700 1 -- 172.21.15.191:0/339926555 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f79ce75340 con 0x55f79cdfb400 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f5c65700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18ac00 0x55f79d0e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f79cdfb800 0x55f79d0e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79cdfbc00 con 0x55f79cdfb800 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f6c67700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f79cdfb800 0x55f79d0e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f5c65700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18ac00 0x55f79d0e3600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55f79d0dd890 tx=0x55f79ceeb800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.411 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.394+0000 7f28f6c67700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f79cdfb800 0x55f79d0e3b80 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 realm 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b000 con 0x55f79cdfb800 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b400 con 0x55f79cdfb800 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b800 con 0x55f79cdfb800 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f79d18bc00 0x55f79d0e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.395+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:5 5.0 5:0267c8b5:::zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b000 con 0x55f79d18bc00 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f6466700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f79d18bc00 0x55f79d0e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f6466700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f79d18bc00 0x55f79d0e4100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 1 ==== osd_op_reply(5 zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b [read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f84bca80 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f84bca80 20 searching for the correct realm 2024-04-27T04:36:30.413 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b400 con 0x55f79d18bc00 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=107b] v33'4 uv4 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f6466700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f79d18b400 0x55f79d0e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.396+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b000 con 0x55f79d18b400 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.397+0000 7f28f5c65700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f79d18b400 0x55f79d0e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.397+0000 7f28f5c65700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f79d18b400 0x55f79d0e4c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.397+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.397+0000 7f28f5c65700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f79d18b000 0x55f79d0e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.397+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.397+0000 7f28f6c67700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f79d18b000 0x55f79d0e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f79d18b000 0x55f79d0e5180 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.398+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b400 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.415 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.399+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b400 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.400+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 5 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=112b] v24'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.401+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 6 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b400 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.402+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 7 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b400 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 5 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.433 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b400 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.403+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18bc00 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 8 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b400 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 7 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 20 RGWRados::pool_iterate: got zone_info.e734f6f6-f5d6-4534-ad88-d40773f7ed1b 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 20 RGWRados::pool_iterate: got zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T04:36:30.434 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.404+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79cdfbc00 con 0x55f79cdfb800 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252400 con 0x55f79cdfb800 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:40 5.1 5:8046bce2:::zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252800 con 0x55f79d18bc00 2024-04-27T04:36:30.435 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 11 ==== osd_op_reply(40 zonegroup_info.29d2c105-c074-4639-b257-47b1060771f9 [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 zone default found 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 4 Realm: () 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 4 ZoneGroup: default (29d2c105-c074-4639-b257-47b1060771f9) 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 4 Zone: default (e734f6f6-f5d6-4534-ad88-d40773f7ed1b) 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 zonegroup default 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.405+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252c00 con 0x55f79d18b000 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 9 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253000 con 0x55f79d18bc00 2024-04-27T04:36:30.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253400 con 0x55f79d18bc00 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 started sync module instance, tier type = 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 20 started zone id=e734f6f6-f5d6-4534-ad88-d40773f7ed1b (name=default) with tier type = 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.406+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253800 con 0x55f79d18b400 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.407+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 8 ==== osd_op_reply(44 notify.0 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.407+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94521980197632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252800 con 0x55f79d18b400 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.407+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253000 con 0x55f79cdfb800 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.408+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94521980197632] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.408+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 15 ==== osd_op_reply(46 notify.1 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.437 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.408+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94521980198784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253400 con 0x55f79cdfb800 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.408+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252400 con 0x55f79d18bc00 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.409+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94521980198784] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.410+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 14 ==== osd_op_reply(48 notify.2 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.410+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94521980199936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252c00 con 0x55f79d18bc00 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.410+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79cdfbc00 con 0x55f79d18b400 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.411+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94521980199936] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.411+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 10 ==== osd_op_reply(50 notify.3 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.411+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94521980201088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d18b800 con 0x55f79d18b400 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.411+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79cdfb800 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.412+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 17 ==== osd_op_reply(52 notify.4 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.412+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94521980201088] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.412+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94521980202240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253800 con 0x55f79cdfb800 2024-04-27T04:36:30.438 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.412+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252800 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.413+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 10 ==== osd_op_reply(54 notify.5 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.413+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94521980202240] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.413+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94521980203392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253c00 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.413+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253000 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.414+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94521980203392] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.414+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 12 ==== osd_op_reply(56 notify.6 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.414+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94521980204544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253400 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.414+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252000 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 13 ==== osd_op_reply(58 notify.7 [create] v33'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94521980204544] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94521980205696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253800 con 0x55f79d18b000 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=0 2024-04-27T04:36:30.439 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=1 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=2 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=3 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=4 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=5 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.415+0000 7f28f84bca80 20 add_watcher() i=6 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.416+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94521980205696] v33'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.416+0000 7f28f84bca80 20 add_watcher() i=7 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.416+0000 7f28f84bca80 2 all 8 watchers are set, enabling cache 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.416+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252800 con 0x55f79cdfb800 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.416+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.416+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94521981788160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253c00 con 0x55f79cdfb800 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.417+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94521981788160] v33'245 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.417+0000 7f28dd434700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:30.440 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.417+0000 7f28f84bca80 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.417+0000 7f28f84bca80 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.417+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.417+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f84bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f84bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ce9c3c0 con 0x55f79d253400 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f5c65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f5c65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41526/0 (socket says 172.21.15.191:41526) 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f5c65700 1 -- 172.21.15.191:0/2735986068 learned_addr learned my addr 172.21.15.191:0/2735986068 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f5c65700 1 -- 172.21.15.191:0/2735986068 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce22b60 con 0x55f79d253400 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28f5c65700 1 --2- 172.21.15.191:0/2735986068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55f79d19f410 tx=0x55f79ceeb980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90883bc67dfa3aad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28dcc33700 1 -- 172.21.15.191:0/2735986068 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79ceb3760 con 0x55f79d253400 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28dcc33700 1 -- 172.21.15.191:0/2735986068 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79d272000 con 0x55f79d253400 2024-04-27T04:36:30.441 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.418+0000 7f28dcc33700 1 -- 172.21.15.191:0/2735986068 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79d274000 con 0x55f79d253400 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 -- 172.21.15.191:0/2735986068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 msgr2=0x55f79d0e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 --2- 172.21.15.191:0/2735986068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55f79d19f410 tx=0x55f79ceeb980 comp rx=0 tx=0).stop 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 -- 172.21.15.191:0/2735986068 shutdown_connections 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 --2- 172.21.15.191:0/2735986068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 -- 172.21.15.191:0/2735986068 >> 172.21.15.191:0/2735986068 conn(0x55f79d253000 msgr2=0x55f79cee8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 -- 172.21.15.191:0/2735986068 shutdown_connections 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 -- 172.21.15.191:0/2735986068 wait complete. 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 Processor -- start 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.419+0000 7f28f84bca80 1 -- start start 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f84bca80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f84bca80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ce9c500 con 0x55f79d253400 2024-04-27T04:36:30.442 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f5c65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f5c65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.191:41534/0 (socket says 172.21.15.191:41534) 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f5c65700 1 -- 172.21.15.191:0/355292847 learned_addr learned my addr 172.21.15.191:0/355292847 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f5c65700 1 -- 172.21.15.191:0/355292847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79ce22d00 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f5c65700 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x55f79d267260 tx=0x55f79d23c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28db430700 1 -- 172.21.15.191:0/355292847 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79d274160 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28db430700 1 -- 172.21.15.191:0/355292847 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f79d272180 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f79ce22340 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28db430700 1 -- 172.21.15.191:0/355292847 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f79d2742c0 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.420+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f79ce23d40 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.421+0000 7f28db430700 1 -- 172.21.15.191:0/355292847 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f79db9db80 con 0x55f79d253400 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.421+0000 7f28db430700 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d2ec000 0x55f79d0e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:36:30.443 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.421+0000 7f28db430700 1 -- 172.21.15.191:0/355292847 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f79ce75500 con 0x55f79d253400 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.421+0000 7f28f6c67700 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d2ec000 0x55f79d0e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.421+0000 7f28f84bca80 5 note: GC not initialized 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.422+0000 7f28f6c67700 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d2ec000 0x55f79d0e5700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f79d1ea1b0 tx=0x55f79d23c5a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.422+0000 7f28d8c2b700 20 reqs_thread_entry: start 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.422+0000 7f28f84bca80 20 init_complete bucket index max shards: 11 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.422+0000 7f28f84bca80 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.422+0000 7f28d6c27700 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 04:37:00 2024 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.422+0000 7f28d6405700 20 reqs_thread_entry: start 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d253800 con 0x55f79d18b000 2024-04-27T04:36:30.444 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 16 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 20 rados->read ofs=0 len=0 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252800 con 0x55f79cdfb800 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 21 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-27T04:36:30.445 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.423+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~439 in=439b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ec400 con 0x55f79d18b000 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 17 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~439] v33'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f84bca80 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f84bca80 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f84bca80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94521981790464 in=627b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ec800 con 0x55f79cdfb800 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 22 ==== watch-notify(notify (1) cookie 94521980202240 notify 141733920787 ret 0) v3 ==== 657+0+0 (crc 0 0 0) 0x55f79d1f7d40 con 0x55f79cdfb800 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f0c5b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920787 cookie 94521980202240 notifier 4908 bl.length()=615 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f0c5b700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f0c5b700 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 23 ==== osd_op_reply(65 notify.4 [notify cookie 94521981790464 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f0c5b700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d35e000 con 0x55f79cdfb800 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 24 ==== watch-notify(notify_complete (2) cookie 94521981790464 notify 141733920787 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f79d2b7ba0 con 0x55f79cdfb800 2024-04-27T04:36:30.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.425+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ecc00 con 0x55f79cdfb800 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.426+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 26 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v33'21 uv21 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.426+0000 7f28f84bca80 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.426+0000 7f28f84bca80 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.426+0000 7f28f84bca80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.426+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94521981790464 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ed000 con 0x55f79d18bc00 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 16 ==== watch-notify(notify (1) cookie 94521980199936 notify 141733920778 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55f79d145d40 con 0x55f79d18bc00 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f0c5b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920778 cookie 94521980199936 notifier 4908 bl.length()=177 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f0c5b700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f0c5b700 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f0c5b700 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d35e400 con 0x55f79d18bc00 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 17 ==== osd_op_reply(68 notify.2 [notify cookie 94521981790464 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 18 ==== watch-notify(notify_complete (2) cookie 94521981790464 notify 141733920778 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f79d2b3a00 con 0x55f79d18bc00 2024-04-27T04:36:30.447 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.427+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 19 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "user_id": "ada", 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "display_name": "Ms. Admin User", 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "email": "", 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "suspended": 0, 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "max_buckets": 0, 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "subusers": [], 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "keys": [ 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.448 INFO:teuthology.orchestra.run.smithi191.stdout: "user": "ada", 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "access_key": "MH1WC2XQ1S8UISFDZC8W", 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "secret_key": "dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG" 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "swift_keys": [], 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": [ 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "buckets", 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "*" 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.449 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "info", 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "read" 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "ratelimit", 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "*" 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "usage", 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "*" 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "users", 2024-04-27T04:36:30.450 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "*" 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: { 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "zone", 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "perm": "*" 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: ], 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "op_mask": "read, write, delete", 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "default_placement": "", 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "default_storage_class": "", 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "placement_tags": [], 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "bucket_quota": { 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:36:30.451 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "user_quota": { 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "enabled": false, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "check_on_raw": false, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size": -1, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "max_size_kb": 0, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "max_objects": -1 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "temp_url_keys": [], 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "rgw", 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout: "mfa_ids": [] 2024-04-27T04:36:30.452 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-04-27T04:36:30.453 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-04-27T04:36:30.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.428+0000 7f28dd434700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T04:36:30.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.428+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94521981788160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ed400 con 0x55f79cdfb800 2024-04-27T04:36:30.458 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.429+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94521981788160] v33'246 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.429+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94521980197632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ed800 con 0x55f79d18b400 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.430+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94521980197632] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.430+0000 7f28f84bca80 20 remove_watcher() i=0 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.430+0000 7f28f84bca80 2 removed watcher, disabling cache 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.430+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94521980198784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2edc00 con 0x55f79cdfb800 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.431+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94521980198784] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.431+0000 7f28f84bca80 20 remove_watcher() i=1 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.431+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94521980199936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ed000 con 0x55f79d18bc00 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.432+0000 7f28f6466700 1 -- 172.21.15.191:0/339926555 <== osd.3 v2:172.21.15.191:6800/41887 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94521980199936] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237200 con 0x55f79d18bc00 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.432+0000 7f28f84bca80 20 remove_watcher() i=2 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.432+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94521980201088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ec400 con 0x55f79d18b400 2024-04-27T04:36:30.459 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.433+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 <== osd.2 v2:172.21.15.191:6816/41888 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94521980201088] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d237440 con 0x55f79d18b400 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.433+0000 7f28f84bca80 20 remove_watcher() i=3 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.433+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94521980202240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ec800 con 0x55f79cdfb800 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.434+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.1 v2:172.21.15.191:6808/41886 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94521980202240] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79cdfb800 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.434+0000 7f28f84bca80 20 remove_watcher() i=4 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.434+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94521980203392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d2ecc00 con 0x55f79d18b000 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.435+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94521980203392] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.435+0000 7f28f84bca80 20 remove_watcher() i=5 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.435+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94521980204544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252400 con 0x55f79d18b000 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.436+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94521980204544] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.436+0000 7f28f84bca80 20 remove_watcher() i=6 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.436+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 --> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94521980205696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f79d252c00 con 0x55f79d18b000 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f6c67700 1 -- 172.21.15.191:0/339926555 <== osd.0 v2:172.21.15.138:6800/39496 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94521980205696] v33'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f79d236b40 con 0x55f79d18b000 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 20 remove_watcher() i=7 2024-04-27T04:36:30.460 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f79d18b000 msgr2=0x55f79d0e5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:6800/39496,v1:172.21.15.138:6801/39496] conn(0x55f79d18b000 0x55f79d0e5180 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f79cdfb800 msgr2=0x55f79d0e3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6808/41886,v1:172.21.15.191:6809/41886] conn(0x55f79cdfb800 0x55f79d0e3b80 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f79d18b400 msgr2=0x55f79d0e4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6816/41888,v1:172.21.15.191:6817/41888] conn(0x55f79d18b400 0x55f79d0e4c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f79d18bc00 msgr2=0x55f79d0e4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6800/41887,v1:172.21.15.191:6801/41887] conn(0x55f79d18bc00 0x55f79d0e4100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18ac00 msgr2=0x55f79d0e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18ac00 0x55f79d0e3600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55f79d0dd890 tx=0x55f79ceeb800 comp rx=0 tx=0).stop 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 msgr2=0x55f79d0e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.461 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 --2- 172.21.15.191:0/339926555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfb400 0x55f79d0e2b00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55f79dc8c420 tx=0x55f79ceebf20 comp rx=0 tx=0).stop 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 reap_dead start 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f5c65700 1 -- 172.21.15.191:0/339926555 reap_dead start 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 shutdown_connections 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 >> 172.21.15.191:0/339926555 conn(0x55f79cdfac00 msgr2=0x55f79cee7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 shutdown_connections 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.437+0000 7f28f84bca80 1 -- 172.21.15.191:0/339926555 wait complete. 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d2ec000 msgr2=0x55f79d0e5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d2ec000 0x55f79d0e5700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f79d1ea1b0 tx=0x55f79d23c5a0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 msgr2=0x55f79d0e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x55f79d267260 tx=0x55f79d23c8a0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 shutdown_connections 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d2ec000 0x55f79d0e5700 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.462 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/355292847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79d253400 0x55f79d0e4680 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 >> 172.21.15.191:0/355292847 conn(0x55f79d253000 msgr2=0x55f79cee8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 shutdown_connections 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/355292847 wait complete. 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18a000 msgr2=0x55f79d0e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18a000 0x55f79d0e3080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55f79dc8cf00 tx=0x55f79d146240 comp rx=0 tx=0).stop 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 msgr2=0x55f79d0e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x55f79dbcbe60 tx=0x55f79ceeba40 comp rx=0 tx=0).stop 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 shutdown_connections 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x55f79d18a000 0x55f79d0e3080 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 --2- 172.21.15.191:0/560341667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f79cdfa800 0x55f79d0e2580 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 >> 172.21.15.191:0/560341667 conn(0x55f79cdfa400 msgr2=0x55f79cee7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 shutdown_connections 2024-04-27T04:36:30.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-04-27T04:36:30.438+0000 7f28f84bca80 1 -- 172.21.15.191:0/560341667 wait complete. 2024-04-27T04:36:30.464 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.464 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.464 ERROR:root:0 2024-04-27T04:36:30.465 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.471 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx000005d31d182ee9dff0e-00662c80ce-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:36:30.472 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-04-27T04:36:30.482 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.483 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-04-27T04:36:30.496 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-04-27T04:36:30.496 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-04-27T04:36:30.508 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-27T04:36:30.508 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-27T04:36:30.515 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-04-27T04:36:30.516 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.519 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.520 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-04-27T04:36:30.524 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.525 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-27T04:36:30.530 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.531 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.534 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.535 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-27T04:36:30.540 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.541 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.544 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.544 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-04-27T04:36:30.552 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}] 2024-04-27T04:36:30.552 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.556 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.556 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-04-27T04:36:30.563 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.563 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.566 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.567 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-04-27T04:36:30.574 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-04-27T04:36:30.574 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.577 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.578 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-04-27T04:36:30.585 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-04-27T04:36:30.586 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.589 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.590 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-04-27T04:36:30.596 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.597 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.600 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.601 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-04-27T04:36:30.605 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.606 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.609 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.609 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-04-27T04:36:30.616 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.616 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.620 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.620 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.624 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T04:36:30.625 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-27T04:36:30.629 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T04:36:30.641 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-27T04:36:30.645 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-04-27T04:36:30.646 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T04:36:30.649 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:36:30.635381Z', 'creation_time': '2024-04-27T04:36:30.632841Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T04:36:30.650 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-27T04:36:30.654 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:36:30.635381Z', 'creation_time': '2024-04-27T04:36:30.632841Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-27T04:36:30.660 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T04:36:30.665 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:36:30.635381Z', 'creation_time': '2024-04-27T04:36:30.632841Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T04:36:30.665 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-04-27T04:36:30.669 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx0000087d9cd0c38e4c05b-00662c80ce-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:36:30.674 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-04-27T04:36:30.681 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.681 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-04-27T04:36:30.689 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:36:30.739 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9'} 2024-04-27T04:36:30.749 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.753 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9'} 2024-04-27T04:36:30.766 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.766 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-04-27T04:36:30.775 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.782 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-04-27T04:36:30.788 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:36:30.789 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T04:36:30.794 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '29d2c105-c074-4639-b257-47b1060771f9', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'marker': 'e734f6f6-f5d6-4534-ad88-d40773f7ed1b.4223.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T04:36:30.757566Z', 'creation_time': '2024-04-27T04:36:30.632841Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T04:37:30.842 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-04-27T04:37:30.861 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'anonymous', 'categories': []}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'bar', 'categories': []}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'bud', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'fud', 'categories': []}, {'bucket': 'myfoo7', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'fud', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}]} 2024-04-27T04:37:30.861 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-04-27T04:37:30.876 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}]}, {'bucket': 'myfoo7', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4}}]} 2024-04-27T04:37:30.876 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-27T04:37:30.881 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}]} 2024-04-27T04:37:30.881 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-04-27T04:37:30.885 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}}]} 2024-04-27T04:37:30.885 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-04-27T04:37:30.890 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T04:37:30.890 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-04-27T04:37:30.894 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}]} 2024-04-27T04:37:30.894 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-04-27T04:37:30.898 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': []}, {'bucket': 'useless-bucket', 'time': '2024-04-27T04:00:00.000000Z', 'epoch': 1714190400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}]} 2024-04-27T04:37:30.899 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-04-27T04:37:30.906 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:37:30.906 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-27T04:37:30.910 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-04-27T04:37:30.911 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-27T04:37:30.919 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:37:30.923 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-27T04:37:30.931 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:37:31.189 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-27T04:37:31.194 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx0000054ab5eafaafe8aac-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.295 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-27T04:37:31.299 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-27T04:37:31.304 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-27T04:37:31.309 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'id': '', 'email': '', 'permission': {'flags': 1}, 'name': '', 'group': 1, 'url_spec': ''}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-27T04:37:31.410 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-04-27T04:37:31.454 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:37:31.455 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-27T04:37:31.460 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-04-27T04:37:31.461 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-27T04:37:31.466 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T04:37:31.476 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-27T04:37:31.480 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000d0320a61c9ca8c5f-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.488 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-04-27T04:37:31.522 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:37:31.522 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T04:37:31.526 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx0000025298ecc82169607-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.526 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-04-27T04:37:31.530 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': '512e09fb-564f-4fb6-aa0b-f541dbeb017d'}]}} 2024-04-27T04:37:31.531 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-04-27T04:37:31.541 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T04:37:31.541 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-27T04:37:31.545 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T04:37:31.545 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-04-27T04:37:31.549 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000b7e562bb09d704df-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.549 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-04-27T04:37:31.553 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000ba9cdcc836553810-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.563 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-27T04:37:31.566 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T04:37:31.567 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-04-27T04:37:31.570 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000bd6e6bb4c30b90d1-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.571 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-04-27T04:37:31.574 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000be501609fe704d4a-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.575 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-04-27T04:37:31.579 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T04:37:31.579 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-04-27T04:37:31.582 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx0000068d57fed8e2f36ed-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.583 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-04-27T04:37:31.586 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000da22ff45f6c20c79-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.587 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-04-27T04:37:31.590 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000964884ee8cfede22-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.590 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-04-27T04:37:31.594 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000125b648012d8e17a-00662c810b-107d-default', 'HostId': '107d-default-default'} 2024-04-27T04:37:31.594 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-04-27T04:37:31.605 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:37:31.605 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-27T04:37:31.609 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-04-27T04:37:31.609 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-04-27T04:37:31.614 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:37:31.614 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-27T04:37:31.617 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-04-27T04:37:31.618 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-04-27T04:37:31.623 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T04:37:31.630 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-27T04:37:31.641 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-04-27T04:37:31.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:31.643 INFO:tasks.rgw.client.0.smithi191.stdout:2024-04-27T04:37:31.641+0000 7fc5b690d700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 46912) UID: 0 2024-04-27T04:37:31.643 INFO:tasks.rgw.client.0.smithi191.stdout:2024-04-27T04:37:31.641+0000 7fc5bab7aa80 -1 shutting down 2024-04-27T04:37:36.869 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-04-27T04:37:37.743 INFO:tasks.rgw.client.0:Stopped 2024-04-27T04:37:37.744 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-04-27T04:37:37.766 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/vault-root-token 2024-04-27T04:37:37.837 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-27T04:37:37.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa385e04700 1 Processor -- start 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa385e04700 1 -- start start 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa385e04700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3800939f0 0x7fa380093dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa385e04700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38006d980 con 0x7fa380094310 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa385e04700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa380095a50 con 0x7fa3800939f0 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa37f7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3800939f0 0x7fa380093dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa37effd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa380094310 0x7fa380129e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39382/0 (socket says 172.21.15.138:39382) 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa37effd700 1 -- 172.21.15.138:0/624325634 learned_addr learned my addr 172.21.15.138:0/624325634 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T04:37:37.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.981+0000 7fa37e7fc700 1 -- 172.21.15.138:0/624325634 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437482741 0 0) 0x7fa38006d980 con 0x7fa380094310 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37e7fc700 1 -- 172.21.15.138:0/624325634 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3680035b0 con 0x7fa380094310 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37f7fe700 1 -- 172.21.15.138:0/624325634 >> v1:172.21.15.138:6789/0 conn(0x7fa380094310 legacy=0x7fa380129e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37f7fe700 1 -- 172.21.15.138:0/624325634 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38006d590 con 0x7fa3800939f0 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37f7fe700 1 --2- 172.21.15.138:0/624325634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa380093dd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa3700097c0 tx=0x7fa370009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5917a934dc6e6a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37e7fc700 1 -- 172.21.15.138:0/624325634 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa370004160 con 0x7fa3800939f0 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37e7fc700 1 -- 172.21.15.138:0/624325634 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3700042c0 con 0x7fa3800939f0 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa37e7fc700 1 -- 172.21.15.138:0/624325634 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa370004580 con 0x7fa3800939f0 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 -- 172.21.15.138:0/624325634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 msgr2=0x7fa380093dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 --2- 172.21.15.138:0/624325634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa380093dd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa3700097c0 tx=0x7fa370009ad0 comp rx=0 tx=0).stop 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 -- 172.21.15.138:0/624325634 shutdown_connections 2024-04-27T04:37:37.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 --2- 172.21.15.138:0/624325634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa380093dd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:37:37.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 -- 172.21.15.138:0/624325634 >> 172.21.15.138:0/624325634 conn(0x7fa380099b60 msgr2=0x7fa380099f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:37:37.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 -- 172.21.15.138:0/624325634 shutdown_connections 2024-04-27T04:37:37.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.982+0000 7fa385e04700 1 -- 172.21.15.138:0/624325634 wait complete. 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.983+0000 7fa385e04700 1 Processor -- start 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.983+0000 7fa385e04700 1 -- start start 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa385e04700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa3801bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa385e04700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380127bf0 con 0x7fa3800939f0 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa3801bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa3801bfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48956/0 (socket says 172.21.15.138:48956) 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa37f7fe700 1 -- 172.21.15.138:0/2742626893 learned_addr learned my addr 172.21.15.138:0/2742626893 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa37f7fe700 1 -- 172.21.15.138:0/2742626893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700094a0 con 0x7fa3800939f0 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa37f7fe700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa3801bfc40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa370009b10 tx=0x7fa3700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:37:37.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa370003db0 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370011630 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa380096fc0 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3700118f0 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.984+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa380127370 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.985+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa370011a50 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.985+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38009cb70 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.985+0000 7fa367fff700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fa36c037780 0x7fa36c039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.986+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fa37004c5c0 con 0x7fa3800939f0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.986+0000 7fa37effd700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fa36c037780 0x7fa36c039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T04:37:37.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.986+0000 7fa37effd700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fa36c037780 0x7fa36c039c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa374000f60 tx=0x7fa3740010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T04:37:37.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:37.990+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa370011d00 con 0x7fa3800939f0 2024-04-27T04:37:38.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.178+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fa380062510 con 0x7fa3800939f0 2024-04-27T04:37:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.181+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa370011d00 con 0x7fa3800939f0 2024-04-27T04:37:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.181+0000 7fa367fff700 1 -- 172.21.15.138:0/2742626893 <== mon.0 v2:172.21.15.138:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7fa37002d3a0 con 0x7fa3800939f0 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fa36c037780 msgr2=0x7fa36c039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fa36c037780 0x7fa36c039c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa374000f60 tx=0x7fa3740010a0 comp rx=0 tx=0).stop 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 msgr2=0x7fa3801bfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa3801bfc40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa370009b10 tx=0x7fa3700047c0 comp rx=0 tx=0).stop 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 shutdown_connections 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.191:6824/41511,v1:172.21.15.191:6825/41511] conn(0x7fa36c037780 0x7fa36c039c40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 --2- 172.21.15.138:0/2742626893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3800939f0 0x7fa3801bfc40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.183+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 >> 172.21.15.138:0/2742626893 conn(0x7fa380099b60 msgr2=0x7fa3801af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.184+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 shutdown_connections 2024-04-27T04:37:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T04:37:38.184+0000 7fa385e04700 1 -- 172.21.15.138:0/2742626893 wait complete. 2024-04-27T04:37:38.197 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-27T04:37:38.197 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-27T04:37:38.197 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-27T04:37:38.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:38.237 INFO:tasks.ceph.osd.0:Stopped 2024-04-27T04:37:38.237 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-27T04:37:38.237 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:38.270 INFO:tasks.ceph.osd.1:Stopped 2024-04-27T04:37:38.270 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-27T04:37:38.270 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:38.338 INFO:tasks.ceph.osd.2:Stopped 2024-04-27T04:37:38.338 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-27T04:37:38.338 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:38.405 INFO:tasks.ceph.osd.3:Stopped 2024-04-27T04:37:38.406 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-27T04:37:38.406 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-27T04:37:38.406 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:38.473 INFO:tasks.ceph.mgr.x:Stopped 2024-04-27T04:37:38.474 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-27T04:37:38.474 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-27T04:37:38.474 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T04:37:38.484 INFO:tasks.ceph.mon.a:Stopped 2024-04-27T04:37:38.484 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-27T04:37:38.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-04-27T04:37:38.552 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T04:37:38.553 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-27T04:37:38.738 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:37:38.738 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-27T04:37:39.034 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:37:39.034 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-27T04:37:39.142 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:37:39.142 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-27T04:37:39.209 INFO:tasks.ceph:Archiving mon data... 2024-04-27T04:37:39.210 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/data/mon.a.tgz 2024-04-27T04:37:39.211 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-27T04:37:39.229 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.k5wVX7z4Al 2024-04-27T04:37:39.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.k5wVX7z4Al 2024-04-27T04:37:39.369 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.k5wVX7z4Al 2024-04-27T04:37:39.456 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.k5wVX7z4Al is 216KB 2024-04-27T04:37:39.549 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.k5wVX7z4Al 2024-04-27T04:37:39.566 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-27T04:37:39.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T04:37:39.608 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T04:37:39.640 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-27T04:37:39.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-27T04:37:39.674 DEBUG:teuthology.orchestra.run.smithi191:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-27T04:37:39.699 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-27T04:37:39.712 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-27T04:37:39.719 INFO:tasks.ceph:Archiving crash dumps... 2024-04-27T04:37:39.721 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/remote/smithi138/crash 2024-04-27T04:37:39.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T04:37:39.772 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/remote/smithi191/crash 2024-04-27T04:37:39.773 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T04:37:39.813 INFO:tasks.ceph:Compressing logs... 2024-04-27T04:37:39.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T04:37:39.817 DEBUG:teuthology.orchestra.run.smithi191:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T04:37:41.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-04-27T04:37:42.486 INFO:tasks.ceph:Archiving logs... 2024-04-27T04:37:42.487 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/remote/smithi138/log 2024-04-27T04:37:42.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T04:37:42.670 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/remote/smithi191/log 2024-04-27T04:37:42.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T04:37:43.098 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T04:37:43.110 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T04:37:43.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T04:37:43.113 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T04:37:43.190 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T04:37:43.202 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T04:37:43.202 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T04:37:43.205 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T04:37:43.221 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2024-04-27T04:37:43.222 INFO:teuthology.orchestra.run.smithi138.stderr:bash: ntpq: command not found 2024-04-27T04:37:43.225 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T04:37:43.225 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-04-27T04:37:43.225 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +648us[ +647us] +/- 52ms 2024-04-27T04:37:43.225 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1945us[-1946us] +/- 53ms 2024-04-27T04:37:43.225 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +1431us[+1431us] +/- 62ms 2024-04-27T04:37:43.226 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:37:43.227 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T04:37:43.227 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-27T04:37:43.227 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +644us[ +646us] +/- 52ms 2024-04-27T04:37:43.227 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -1949us[-1947us] +/- 53ms 2024-04-27T04:37:43.227 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +1423us[+1425us] +/- 62ms 2024-04-27T04:37:43.227 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T04:37:43.228 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T04:37:43.239 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T04:37:43.240 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T04:37:43.255 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T04:37:43.334 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T04:37:43.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T04:37:43.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:37:43.427 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T04:37:43.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:37:43.456 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T04:37:43.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T04:37:43.480 INFO:teuthology.task.internal:Duration was 922.956166 seconds 2024-04-27T04:37:43.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T04:37:43.492 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T04:37:43.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T04:37:43.495 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T04:37:43.540 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T04:37:43.542 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T04:37:43.770 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T04:37:43.770 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T04:37:43.771 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T04:37:43.793 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-04-27T04:37:43.793 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T04:37:43.814 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T04:37:43.814 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T04:37:43.835 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T04:37:43.895 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T04:37:43.896 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T04:37:43.917 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T04:37:44.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T04:37:44.014 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T04:37:44.014 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T04:37:44.041 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T04:37:44.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T04:37:44.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:37:44.083 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:37:44.107 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-27T04:37:44.134 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-04-27T04:37:44.160 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:37:44.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:37:44.190 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T04:37:44.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T04:37:44.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T04:37:44.228 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T04:37:44.229 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/remote/smithi138 2024-04-27T04:37:44.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T04:37:44.270 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675699/remote/smithi191 2024-04-27T04:37:44.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T04:37:44.308 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T04:37:44.309 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T04:37:44.311 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T04:37:44.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T04:37:44.372 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T04:37:44.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T04:37:44.389 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T04:37:44.390 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T04:37:44.392 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T04:37:44.407 INFO:teuthology.orchestra.run.smithi138.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 04:37 /home/ubuntu/cephtest 2024-04-27T04:37:44.417 INFO:teuthology.orchestra.run.smithi191.stdout: 526076 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 04:37 /home/ubuntu/cephtest 2024-04-27T04:37:44.418 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T04:37:44.430 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T04:37:44.455 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} duration: 922.9561655521393 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-27T04:37:44.456 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T04:37:44.516 INFO:teuthology.run:pass